You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Robert Fitzpatrick <li...@webtent.net> on 2007/08/16 22:15:24 UTC
Suggested botnet rule scores
I have some spam hitting some users pretty hard while just falling short
of the kill level, see below. Seems if I was using Botnet a little more,
it would help. I remember when we installed the Botnet rules, they were
too aggressive with lots of complaints stemming from mis-configured dns,
yada, yada, yada...so we disabled all but nodns. Now, it seems we may be
catching some stuff if we score them just a bit. Wondering what score
settings others are using for Botnet or are you able to kill these
messages without it?
http://esmtp.webtent.net/mail1.txt
Content analysis details: (4.2 points, 5.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
0.0 BOTNET_CLIENTWORDS Hostname contains client-like substrings
[botnet_clientwords,ip=72.51.59.60,rdns=60.bo.static.symmetrixns1.com]
0.0 BOTNET Relay might be a spambot or virusbot
[botnet0.7,ip=72.51.59.60,hostname=60.bo.static.symmetrixns1.com,maildomain=sitores.villanously.com,client,clientwords]
0.0 BOTNET_CLIENT Relay has a client-like hostname
[botnet_client,ip=72.51.59.60,hostname=60.bo.static.symmetrixns1.com,clientwords]
0.0 ACT_NOW_CAPS BODY: Talks about 'acting now' with capitals
2.8 PYZOR_CHECK Listed in Pyzor (http://pyzor.sf.net/)
1.4 DCC_CHECK Listed in DCC (http://rhyolite.com/anti-spam/dcc/)
0.0 DIGEST_MULTIPLE Message hits more than one network digest check
Thanks for any help!
--
Robert
Re: Suggested botnet rule scores
Posted by John Rudd <jr...@ucsc.edu>.
Jari Fredriksson wrote:
>> Jari Fredriksson wrote on Fri, 17 Aug 2007 01:11:37 +0300:
>>
>>> But if I were an ISP I could not use it. Impossible.
>>> Totally impossible.
>> because ... ?
>>
>> Kai
>
> Because there is always some friends of some customers using a local linux with a local mail server without smart host.
>
And they shouldn't be required to have proper DNS and follow best
practices because?
Re: Suggested botnet rule scores
Posted by Kai Schaetzl <ma...@conactive.com>.
Jari Fredriksson wrote on Fri, 17 Aug 2007 18:39:13 +0300:
> It's common practise here for households, but not for business users. Actually roaming
business users with their lap tops actually need something like a "personal mail server",
no, they don't. Not at all.
> and there are such products for windows too.
It doesn't matter if such products exist. Many providers simply don't accept mail from known
dynamic IP space, for good reasons.
Kai
--
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
Re: Suggested botnet rule scores
Posted by Nix <ni...@esperi.org.uk>.
On 18 Aug 2007, Magnus Holmgren said:
> On Saturday 18 August 2007 16:14, Nix wrote:
>> On 17 Aug 2007, Robert Fitzpatrick verbalised:
>> > ISP's are blocking port 25 from anything but their own stuff, especially
>> > dial-up.
>>
>> Mine blocks until you prove you're competent (or post a bond: I did the
>> former) and gets really pissed if you then turn into a spamming monster.
>> It seems to work.
>
> What did you have to do to prove you're competent?
In my case, `know the ISP's sysadmin'. (It's clued userbase has quite a
collegial atmosphere.)
Re: Suggested botnet rule scores
Posted by Magnus Holmgren <ho...@lysator.liu.se>.
On Saturday 18 August 2007 16:14, Nix wrote:
> On 17 Aug 2007, Robert Fitzpatrick verbalised:
> > ISP's are blocking port 25 from anything but their own stuff, especially
> > dial-up.
>
> Mine blocks until you prove you're competent (or post a bond: I did the
> former) and gets really pissed if you then turn into a spamming monster.
> It seems to work.
What did you have to do to prove you're competent?
--
Magnus Holmgren holmgren@lysator.liu.se
(No Cc of list mail needed, thanks)
"Exim is better at being younger, whereas sendmail is better for
Scrabble (50 point bonus for clearing your rack)" -- Dave Evans
Re: Suggested botnet rule scores
Posted by Jerry Durand <jd...@interstellar.com>.
At 08:11 AM 8/18/2007, Robert Fitzpatrick wrote:
>Botnet is designed to
>combat you.
Along with several black lists. Two of the lists we use do there
best to block dynamic servers.
Note, we are on a dynamic address, but send through our ISPs server
with AUTH. If we had any trouble with their server, we could send
through our backup mail server (also a fixed IP address). Rarely has
this been a problem, only once on this list when I left a name in the
REPLY_ALL and once on another list where a major ISP was rejecting
the entire list anytime ANYONE from a dynamic address posted a message.
--
Jerry Durand, Durand Interstellar, Inc. www.interstellar.com
tel: +1 408 356-3886, USA toll free: 1 866 356-3886
Skype: jerrydurand
Re: Suggested botnet rule scores
Posted by Nix <ni...@esperi.org.uk>.
On 22 Aug 2007, John Rudd spake thusly:
> Nix wrote:
>> My ISP doesn't give me that option (well, OK, it probably gives *me*
>> that option because I can bug the ISP's technical director, but not
>> people who've posted bonds). I'd venture to guess that the vast majority of
>> small business UK ISPs, even those that do not provide useful outbound relaying
>> MTAs, do not delegate rDNS to individual users.
>
> And they can't set one of the MX records, or A records, for their mail domain to be the same as that of the static IP address their
> static IP address?
>
> Because EITHER one of those things will trigger an exception for Botnet.
Oh, right, so botnot only triggers if you're sending from something
that isn't an MX *and* satisfies one of the other criteria?
That's sensible, and I hadn't thought of it, and I'd also brilliantly
managed to overlook it repeatedly when wandering through the botnet
code. (God knows how. Insufficient coffee, probably.)
There are sometimes reasons for a host without an MX to send mail, but
it's bloody rare outside of big clusters (i.e. not boxes fronting for
little networks), and I can see no reason why anyone can't get a low-
priority MX pointing at them even if they can't run an MTA on it (no
harm will be done in that case, of course).
Re: Suggested botnet rule scores
Posted by John Rudd <jr...@ucsc.edu>.
Nix wrote:
> On 21 Aug 2007, Kai Schaetzl said:
>
>> Nix wrote on Tue, 21 Aug 2007 09:26:18 +0100:
>>
>>> It's not dynamic, but Botnet isn't just looking for dynamic IPed hosts, but
>>> also hosts with e.g. the string `adsl' in its rDNS, even if that host happens
>>> to have a static assignment.
>> Well, if it's static they can give you rDNS and you can use a hostname of your
>> choice for A and PTR.
>
> My ISP doesn't give me that option (well, OK, it probably gives *me*
> that option because I can bug the ISP's technical director, but not
> people who've posted bonds). I'd venture to guess that the vast majority of
> small business UK ISPs, even those that do not provide useful outbound relaying
> MTAs, do not delegate rDNS to individual users.
And they can't set one of the MX records, or A records, for their mail
domain to be the same as that of the static IP address their static IP
address?
Because EITHER one of those things will trigger an exception for Botnet.
> (Of course, this is all sophistry to an extent: right now the vast
> majority of mail sent directly from ADSL lines *is* probably sent from
> botted hosts. :( )
Right... which pretty much confirms the motivation behind Botnet.
Yes, it's a _policy_ based mechanism instead of a "spam detection"
mechanims, but the sad truth is that it works MUCH more often than it fails.
Re: Suggested botnet rule scores
Posted by Nix <ni...@esperi.org.uk>.
On 21 Aug 2007, Kai Schaetzl said:
> Nix wrote on Tue, 21 Aug 2007 09:26:18 +0100:
>
>> It's not dynamic, but Botnet isn't just looking for dynamic IPed hosts, but
>> also hosts with e.g. the string `adsl' in its rDNS, even if that host happens
>> to have a static assignment.
>
> Well, if it's static they can give you rDNS and you can use a hostname of your
> choice for A and PTR.
My ISP doesn't give me that option (well, OK, it probably gives *me*
that option because I can bug the ISP's technical director, but not
people who've posted bonds). I'd venture to guess that the vast majority of
small business UK ISPs, even those that do not provide useful outbound relaying
MTAs, do not delegate rDNS to individual users.
(Of course, this is all sophistry to an extent: right now the vast
majority of mail sent directly from ADSL lines *is* probably sent from
botted hosts. :( )
Re: Suggested botnet rule scores
Posted by Kai Schaetzl <ma...@conactive.com>.
Nix wrote on Tue, 21 Aug 2007 09:26:18 +0100:
> It's not dynamic, but Botnet isn't just looking for dynamic IPed hosts, but
> also hosts with e.g. the string `adsl' in its rDNS, even if that host happens
> to have a static assignment.
Well, if it's static they can give you rDNS and you can use a hostname of your
choice for A and PTR.
Kai
--
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
Re: Suggested botnet rule scores
Posted by Nix <ni...@esperi.org.uk>.
On 18 Aug 2007, Kai Schaetzl stated:
> Nix wrote on Sat, 18 Aug 2007 17:35:20 +0100:
>
>> Competent ISPs give you rDNS. (Really good ones delegate your rDNS to
>> you.)
>
> So, your ISP is not competent? How would they give specific rDNS to
> dynamic IP addresses, anyway?
It's not dynamic, but Botnet isn't just looking for dynamic IPed hosts, but
also hosts with e.g. the string `adsl' in its rDNS, even if that host happens
to have a static assignment.
Re: Suggested botnet rule scores
Posted by Kai Schaetzl <ma...@conactive.com>.
Nix wrote on Sat, 18 Aug 2007 17:35:20 +0100:
> Competent ISPs give you rDNS. (Really good ones delegate your rDNS to
> you.)
So, your ISP is not competent? How would they give specific rDNS to
dynamic IP addresses, anyway?
Kai
--
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
Re: Suggested botnet rule scores
Posted by Nix <ni...@esperi.org.uk>.
On 18 Aug 2007, Robert Fitzpatrick spake thusly:
> On Sat, 2007-08-18 at 15:14 +0100, Nix wrote:
>> On 17 Aug 2007, Robert Fitzpatrick verbalised:
>> > Worms and spam have made it impossible for users to use their own
>> > personal mail servers.
>>
>> Really? Fascinating, I'm doing the impossible. I had no idea.
>
> Correction, normal novice users that don't have to bargain with ISP's or
> know anything about spam techniques.
Sure.
> It is only a matter of time before
> somebody blocks you, if they haven't already. Botnet is designed to
> combat you.
Er, no, Botnet should be designed to combat *spammers*. If it's actually
designed to combat clued non-spammers, then anyone deploying it is a
moron.
Thankfully this doesn't actually seem to be the case.
>> Mine blocks until you prove you're competent (or post a bond: I did the
>> former) and gets really pissed if you then turn into a spamming monster.
>> It seems to work.
>
> And you really want to deal with your ISP instead of using a valid mail
> server?
I do use a `valid mail server'. It just isn't provided by my ISP (they
don't actually provide one other than a low-priority inbound MX in case
I have major line problems). This is a reasonably common setup among
ISPs for clued users in the UK.
> How do you handle rDNS?
Competent ISPs give you rDNS. (Really good ones delegate your rDNS to
you.)
Re: Suggested botnet rule scores
Posted by Robert Fitzpatrick <li...@webtent.net>.
On Sat, 2007-08-18 at 15:14 +0100, Nix wrote:
> On 17 Aug 2007, Robert Fitzpatrick verbalised:
> > Worms and spam have made it impossible for users to use their own
> > personal mail servers.
>
> Really? Fascinating, I'm doing the impossible. I had no idea.
Correction, normal novice users that don't have to bargain with ISP's or
know anything about spam techniques. It is only a matter of time before
somebody blocks you, if they haven't already. Botnet is designed to
combat you.
>
> > More and more
> > ISP's are blocking port 25 from anything but their own stuff, especially
> > dial-up.
>
> Mine blocks until you prove you're competent (or post a bond: I did the
> former) and gets really pissed if you then turn into a spamming monster.
> It seems to work.
And you really want to deal with your ISP instead of using a valid mail
server? How do you handle rDNS?
--
Robert
Re: Suggested botnet rule scores
Posted by Kai Schaetzl <ma...@conactive.com>.
Nix wrote on Tue, 21 Aug 2007 23:24:23 +0100:
> (Personally I'd prefer that *no* single rule could push a mail more than
> halfway towards spamminess...)
Absolutely agreed, with a few exceptions, like Bayes_99 :-)
Kai
--
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
Re: Suggested botnet rule scores
Posted by Nix <ni...@esperi.org.uk>.
On 21 Aug 2007, Kai Schaetzl outgrape:
> Nix wrote on Tue, 21 Aug 2007 09:27:11 +0100:
>
>> If anybody is really so stupid as to unconditionally block mail from
>> hosts merely because of string matching in their rDNS, I'm not sure they
>> *deserve* to see any email...
>
> No, it's stupid to send mail from "adsl" named ranges if you want to get
> your mail read. I think we can drop this discussion. It doesn't matter
> what you or I think, I just told you what's common practice and keeps
> getting "commoner".
I'm just pointing out, *again*, that this should not be the sole
criterion. I don't think I've had any spam that's scored less than about
4.0 in a week or so: you don't need botnet to contribute much to push
that above 5.0.
(Personally I'd prefer that *no* single rule could push a mail more than
halfway towards spamminess...)
Re: Suggested botnet rule scores
Posted by Kai Schaetzl <ma...@conactive.com>.
Nix wrote on Tue, 21 Aug 2007 09:27:11 +0100:
> If anybody is really so stupid as to unconditionally block mail from
> hosts merely because of string matching in their rDNS, I'm not sure they
> *deserve* to see any email...
No, it's stupid to send mail from "adsl" named ranges if you want to get
your mail read. I think we can drop this discussion. It doesn't matter
what you or I think, I just told you what's common practice and keeps
getting "commoner".
Kai
--
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
Re: Suggested botnet rule scores
Posted by Nix <ni...@esperi.org.uk>.
On 18 Aug 2007, Kai Schaetzl said:
> Nix wrote on Sat, 18 Aug 2007 15:14:53 +0100:
>
>> > Worms and spam have made it impossible for users to use their own
>> > personal mail servers.
>>
>> Really? Fascinating, I'm doing the impossible. I had no idea.
>
> You should not read that literally. You can, of course do that. But many
> providers will not let you out and many won't let you in. You are lucky
> that this IP isn't in any dynamic list yet, but it's got "adsl" in the
> hostname, so many will block it that way. So, in general, if you want to
> be sure that your mail gets accepted you have to use your smarthost. Or
> you gamble. Your choice.
If anybody is really so stupid as to unconditionally block mail from
hosts merely because of string matching in their rDNS, I'm not sure they
*deserve* to see any email...
Re: Suggested botnet rule scores
Posted by Kai Schaetzl <ma...@conactive.com>.
Nix wrote on Sat, 18 Aug 2007 15:14:53 +0100:
> > Worms and spam have made it impossible for users to use their own
> > personal mail servers.
>
> Really? Fascinating, I'm doing the impossible. I had no idea.
You should not read that literally. You can, of course do that. But many
providers will not let you out and many won't let you in. You are lucky
that this IP isn't in any dynamic list yet, but it's got "adsl" in the
hostname, so many will block it that way. So, in general, if you want to
be sure that your mail gets accepted you have to use your smarthost. Or
you gamble. Your choice.
Kai
--
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
Re: Suggested botnet rule scores
Posted by Nix <ni...@esperi.org.uk>.
On 17 Aug 2007, Robert Fitzpatrick verbalised:
> Worms and spam have made it impossible for users to use their own
> personal mail servers.
Really? Fascinating, I'm doing the impossible. I had no idea.
> More and more
> ISP's are blocking port 25 from anything but their own stuff, especially
> dial-up.
Mine blocks until you prove you're competent (or post a bond: I did the
former) and gets really pissed if you then turn into a spamming monster.
It seems to work.
Re: Suggested botnet rule scores
Posted by Kai Schaetzl <ma...@conactive.com>.
John Thompson wrote on Mon, 20 Aug 2007 21:36:51 -0500:
> Indeed. But some people have a religious objection to all things google,
> so I hesitate to recommend it as a universal solution.
Misunderstanding. I meant to say that you do not need a Google Mail account
for this. That is why it is an "easy solution for most users". As I said:
"nothing special".
Kai
--
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
Re: Suggested botnet rule scores
Posted by John Thompson <jo...@vector.os2.dhs.org>.
On 2007-08-20, Kai Schaetzl <ma...@conactive.com> wrote:
> John Thompson wrote on Sun, 19 Aug 2007 15:30:59 -0500:
>
>> An easy solution for laptop users with a gmail account is to simply use
>> gmails' SMTP service,
> That is an easy solution for most users, gmail or not. Gmail is really
> nothing special.
Indeed. But some people have a religious objection to all things google,
so I hesitate to recommend it as a universal solution.
--
John (john@os2.dhs.org)
Re: Suggested botnet rule scores
Posted by Kai Schaetzl <ma...@conactive.com>.
John Thompson wrote on Sun, 19 Aug 2007 15:30:59 -0500:
> An easy solution for laptop users with a gmail account is to simply use
> gmails' SMTP service,
That is an easy solution for most users, gmail or not. Gmail is really
nothing special.
Kai
--
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
Re: Suggested botnet rule scores
Posted by John Thompson <Jo...@new.rr.com>.
Robert Fitzpatrick wrote:
> Worms and spam have made it impossible for users to use their own
> personal mail servers. We block any outgoing mail on any managed
> firewall on port 25 other than authorized ESMTP servers. More and more
> ISP's are blocking port 25 from anything but their own stuff, especially
> dial-up. Laptop users should use an ISP that allows him to use their
> servers or his own, then use a mail provider that support SMTP AUTH.
An easy solution for laptop users with a gmail account is to simply use
gmails' SMTP service, which uses TLS by default for transport.
--
-John Thompson (john@os2.dhs.org)
Appleton WI USA
Re: Suggested botnet rule scores
Posted by Robert Fitzpatrick <li...@webtent.net>.
On Fri, 2007-08-17 at 18:39 +0300, Jari Fredriksson wrote:
> > 2. many ISPs block connections from dynamic IPs, anyway,
> > this is actually common practice.
> >
>
> It's common practise here for households, but not for business users.
> Actually roaming business users with their lap tops actually need
> something like a "personal mail server", and there are such products
> for windows too.
Worms and spam have made it impossible for users to use their own
personal mail servers. We block any outgoing mail on any managed
firewall on port 25 other than authorized ESMTP servers. More and more
ISP's are blocking port 25 from anything but their own stuff, especially
dial-up. Laptop users should use an ISP that allows him to use their
servers or his own, then use a mail provider that support SMTP AUTH.
--
Robert
Re: Suggested botnet rule scores
Posted by Jari Fredriksson <ja...@iki.fi>.
> Jari Fredriksson wrote on Fri, 17 Aug 2007 14:39:44 +0300:
>
>> Because there is always some friends of some customers
>> using a local linux with a local mail server without
>> smart host.
>
> And that is a problem?
> 1. you can adjust scoring
That's true, I didn't think about it. So true.
> 2. many ISPs block connections from dynamic IPs, anyway,
> this is actually common practice.
>
It's common practise here for households, but not for business users. Actually roaming business users with their lap tops actually need something like a "personal mail server", and there are such products for windows too.
Re: Suggested botnet rule scores
Posted by Kai Schaetzl <ma...@conactive.com>.
Jari Fredriksson wrote on Fri, 17 Aug 2007 14:39:44 +0300:
> Because there is always some friends of some customers using a local linux with a local mail server without smart host.
And that is a problem?
1. you can adjust scoring
2. many ISPs block connections from dynamic IPs, anyway, this is actually common practice.
Kai
--
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
Re: Suggested botnet rule scores
Posted by Jari Fredriksson <ja...@iki.fi>.
> Jari Fredriksson wrote on Fri, 17 Aug 2007 01:11:37 +0300:
>
>> But if I were an ISP I could not use it. Impossible.
>> Totally impossible.
>
> because ... ?
>
> Kai
Because there is always some friends of some customers using a local linux with a local mail server without smart host.
Re: Suggested botnet rule scores
Posted by Kai Schaetzl <ma...@conactive.com>.
Jari Fredriksson wrote on Fri, 17 Aug 2007 01:11:37 +0300:
> But if I were an ISP I could not use it. Impossible. Totally impossible.
because ... ?
Kai
--
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
Re: Suggested botnet rule scores
Posted by Loren Wilton <lw...@earthlink.net>.
> Kai Schaetzl wrote:
>
> I see. My pov on quarantine is that as most as possible it should not need
> human review. Clients should be bothered as few as possible. I don't
> reject
> any spam, it's all put in the quarantine. If it scores between 5 and 6
> users get a notice, if it is higher they don't.
FWIW, I adjusted the SA config to put the score into the front of the mail
subject on spams. Then when I glance in the spam folder I just click the
subject header to sort by subject, and quickly scan through the dozen or so
really low-scoring messages and either pull out or just delete the ones that
were "valid" commercial mail. Takes about 20 seconds, and if I'm bored I
might do it a couple times a day.
Putting the score there is a REAL help. Before I did that I remember it
used to take me quite a lot longer to be sure I didn't have something real
in the spam folder. Not is ti fast.
Loren
Re: Suggested botnet rule scores
Posted by John Rudd <jr...@ucsc.edu>.
Kai Schaetzl wrote:
> John Rudd wrote on Fri, 17 Aug 2007 09:01:27 -0700:
>
>> 3) you can eliminate the false positives entirely by setting the score
>> to 4.0, because all of the false positives we've come across were in the
>> range 5.0 <= score < 6 (actually, smaller than 6, but definitely 6 works
>> there).
>
> That sounds good. Will try after I have some results on the 2.0 score.
> "Unfortunately" I'm not getting much spam on my test machine that could get
> hit by Botnet. Ahm, do you use any of the other "minor" rules with small
> scores or do you keep them all at 0 as in the provided BotNet.cf?
>
I keep them at 0, just like in the default cf file.
Re: Suggested botnet rule scores
Posted by Kai Schaetzl <ma...@conactive.com>.
John Rudd wrote on Fri, 17 Aug 2007 09:01:27 -0700:
> It's deliberately a 5.0 because the purpose is to flag all such messages
> for human review/quarantine (and there's a small assumption there that
> no rational human being is trashing or rejecting messages at a score in
> the range of 5 to 6, or even 5 to 10). Botnet isn't so much saying
> "This message is SPAM!!!", as it is saying "This message requires human
> review".
I see. My pov on quarantine is that as most as possible it should not need
human review. Clients should be bothered as few as possible. I don't reject
any spam, it's all put in the quarantine. If it scores between 5 and 6
users get a notice, if it is higher they don't.
Thanks for the insight on the statistics!
> 3) you can eliminate the false positives entirely by setting the score
> to 4.0, because all of the false positives we've come across were in the
> range 5.0 <= score < 6 (actually, smaller than 6, but definitely 6 works
> there).
That sounds good. Will try after I have some results on the 2.0 score.
"Unfortunately" I'm not getting much spam on my test machine that could get
hit by Botnet. Ahm, do you use any of the other "minor" rules with small
scores or do you keep them all at 0 as in the provided BotNet.cf?
Kai
--
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
Re: Suggested botnet rule scores
Posted by Robert Fitzpatrick <li...@webtent.net>.
On Fri, 2007-08-17 at 09:01 -0700, John Rudd wrote:
> Over the last 9 months, my observation has been that, on a million-ish
> message per day system:
>
> 1) aprox. 1% of Botnet marked messages are false positives
>
> 2) you can reduce false positives from Botnet by 66% by just dropping
> the score to 4.99, because the vast majority of false positives are
> scoring in the range 5 <= score < 5.01
>
> 3) you can eliminate the false positives entirely by setting the score
> to 4.0, because all of the false positives we've come across were in the
> range 5.0 <= score < 6 (actually, smaller than 6, but definitely 6 works
> there).
>
> And, anecdotally, while I'm going to keep using the 5.0 score at home,
> at work the campus email teem has decided to lower it to 4.0 for now (as
> soon as our change management process approves the change), and possibly
> adjust it back up toward 4.9 or 4.99 if that's letting through too many
> low scoring spam messages. (my suggestion was 4.99 and further adjust
> downward as necessary, but the group decided to go to 4.0 now and adjust
> back up if necessary)
Yes, we run nordns at 4.5 with no problem, works well, but we got so
many poorly configured BADNS, we had to drop that and everything else.
Almost any business with its own mail server had the standard ISP IP
notation with static or something. We had to add many IP's to trusted
networks? Is there any way to take that from file. We keep many IPs in
postfix, SA, amavisd-new and possibly Botnet. The words were getting hit
too, that is why maybe I think I need to just tweak my words list since
we're an ISP? Any good working words list out there for an ISP? Thanks.
--
Robert
Re: Suggested botnet rule scores
Posted by John Rudd <jr...@ucsc.edu>.
Kai Schaetzl wrote:
> Robert Fitzpatrick wrote on Fri, 17 Aug 2007 08:56:33 -0400:
>
>> Well, like I said, we had big problems using anything in Botnet except
>> nordns.
>
> That's why everything except the main BOTNET is set to 0 I guess ;-) You
> have to check for yourself if it fits or not. I just enabled a few (using
> a score of 1.) and lowered the main BOTNET score from 5.0 to 2.0. I think
> 5 is much too high as a default, this should be changed. Or maybe it's
> deliberate, so people don't just drop the files on their system without
> reading botnet.txt and botnet.variants.txt :-)
It's deliberately a 5.0 because the purpose is to flag all such messages
for human review/quarantine (and there's a small assumption there that
no rational human being is trashing or rejecting messages at a score in
the range of 5 to 6, or even 5 to 10). Botnet isn't so much saying
"This message is SPAM!!!", as it is saying "This message requires human
review".
Over the last 9 months, my observation has been that, on a million-ish
message per day system:
1) aprox. 1% of Botnet marked messages are false positives
2) you can reduce false positives from Botnet by 66% by just dropping
the score to 4.99, because the vast majority of false positives are
scoring in the range 5 <= score < 5.01
3) you can eliminate the false positives entirely by setting the score
to 4.0, because all of the false positives we've come across were in the
range 5.0 <= score < 6 (actually, smaller than 6, but definitely 6 works
there).
And, anecdotally, while I'm going to keep using the 5.0 score at home,
at work the campus email teem has decided to lower it to 4.0 for now (as
soon as our change management process approves the change), and possibly
adjust it back up toward 4.9 or 4.99 if that's letting through too many
low scoring spam messages. (my suggestion was 4.99 and further adjust
downward as necessary, but the group decided to go to 4.0 now and adjust
back up if necessary)
Re: Suggested botnet rule scores
Posted by Robert Fitzpatrick <li...@webtent.net>.
On Fri, 2007-08-17 at 16:31 +0200, Kai Schaetzl wrote:
> Robert Fitzpatrick wrote on Fri, 17 Aug 2007 08:56:33 -0400:
>
> > Well, like I said, we had big problems using anything in Botnet except
> > nordns.
>
> That's why everything except the main BOTNET is set to 0 I guess ;-) You
> have to check for yourself if it fits or not. I just enabled a few (using
> a score of 1.) and lowered the main BOTNET score from 5.0 to 2.0. I think
> 5 is much too high as a default, this should be changed. Or maybe it's
> deliberate, so people don't just drop the files on their system without
> reading botnet.txt and botnet.variants.txt :-)
>
Yes, we also cut the nordns score to 4.5, been working well since we did
that during that initial setup, now going to try some other things :)
*thanks to all for the suggestions*
--
Robert
Re: Suggested botnet rule scores
Posted by Kai Schaetzl <ma...@conactive.com>.
Robert Fitzpatrick wrote on Fri, 17 Aug 2007 08:56:33 -0400:
> Well, like I said, we had big problems using anything in Botnet except
> nordns.
That's why everything except the main BOTNET is set to 0 I guess ;-) You
have to check for yourself if it fits or not. I just enabled a few (using
a score of 1.) and lowered the main BOTNET score from 5.0 to 2.0. I think
5 is much too high as a default, this should be changed. Or maybe it's
deliberate, so people don't just drop the files on their system without
reading botnet.txt and botnet.variants.txt :-)
Kai
--
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
Re: Suggested botnet rule scores
Posted by Henrik Krohns <he...@hege.li>.
On Fri, Aug 17, 2007 at 08:47:38AM -0700, John Rudd wrote:
> Henrik Krohns wrote:
>
>> If you want a simple solution, you can try http://sa.hege.li/ for BadRelay
>> plugin.
>
> BadRelay makes a fairly fatal assumption: The MTA put the rdns into the
> Received header. I know of 2 MTAs that don't do that (they just put the IP
> address in, without the rdns name). If you're using one of those MTAs,
> then I'll bet you're going to get lots of BadRelay false positives ... just
> like the SA 3.2.1 rule for checking for no-rdns gets lots of false
I don't make any worse assumptions than SA. If it doesn't work, it doesn't
work, get a decend MTA (works for me). I don't see any point having
unnecessary stuff like Net::DNS and dealing with timeout issues, since I
don't need any special checks. I just want to check rdns.
Cheers,
Henrik
Re: Suggested botnet rule scores
Posted by John Rudd <jr...@ucsc.edu>.
Henrik Krohns wrote:
>
> If you want a simple solution, you can try http://sa.hege.li/ for BadRelay
> plugin.
>
BadRelay makes a fairly fatal assumption: The MTA put the rdns into the
Received header. I know of 2 MTAs that don't do that (they just put the
IP address in, without the rdns name). If you're using one of those
MTAs, then I'll bet you're going to get lots of BadRelay false positives
... just like the SA 3.2.1 rule for checking for no-rdns gets lots of
false positives, for the same reason. That's why Botnet, by default,
does an actual rdns lookup on the IP address: so it can remain MTA agnostic.
And, if your MTA did do the rdns lookup, and you've got a sane MTA set
up (local caching name server that retains the lookup for more than a
couple minutes), then the information should still be in the cache when
the plugin does its lookup. That makes the BadRelay attempt at an
optimization into something fairly moot.
Re: Suggested botnet rule scores
Posted by Paweł Tęcza <pt...@uw.edu.pl>.
Henrik Krohns <he...@hege.li> writes:
[...]
> If you want a simple solution, you can try http://sa.hege.li/ for BadRelay
> plugin.
Interesting license... ;)
Have a nice day,
Pawel
Re: Suggested botnet rule scores
Posted by Henrik Krohns <he...@hege.li>.
On Fri, Aug 17, 2007 at 08:56:33AM -0400, Robert Fitzpatrick wrote:
> On Thu, 2007-08-16 at 17:47 -0500, René Berber wrote:
> > Jari Fredriksson wrote:
> >
> > > Botnet is bad AFAIK bad for anyone running an ISP or so.
> > >
> > > I'm a lone one and I know that nobody sending me email is not using a Linux
> > > box with his own server, so I can drop all mail from dynamic dns or no rdns
> > > at all.
> > >
> > > I do whitelist all mailling lists as well, they never see SA.
> > >
> > > In my position, Botnet is good. But if I were an ISP I could not use it.
> > > Impossible. Totally impossible.
> >
> > You never tried, nor need to, and say it is impossible? Not true (have you
> > heard of the trusted_networks setting), it is possible and any ISP who uses SA
> > would gain by using it.
> >
> > The work Botnet does is similar to graylists, a good one stops suspicious mail
> > servers for a while, if they insist they'll pass the graylist and get scored by
> > Botnet, how much you score them is your choice.
>
> Well, like I said, we had big problems using anything in Botnet except
> nordns. Does anyone have a good words list I could try? I have set
> BOTNET_CLIENT to 1.0 and that seems to start killing these messages. I
> also have everything else set to 0 except BOTNET_NORDNS at 4.5. Does all
> the other settings being zero effect my BOTNET_CLIENT scores or will it
> continue to calculate the BOTNET_CLIENTWORDS, etc, as part of
> BOTNET_CLIENT?
If you want a simple solution, you can try http://sa.hege.li/ for BadRelay
plugin.
Re: Suggested botnet rule scores
Posted by Robert Fitzpatrick <li...@webtent.net>.
On Thu, 2007-08-16 at 17:47 -0500, René Berber wrote:
> Jari Fredriksson wrote:
>
> > Botnet is bad AFAIK bad for anyone running an ISP or so.
> >
> > I'm a lone one and I know that nobody sending me email is not using a Linux
> > box with his own server, so I can drop all mail from dynamic dns or no rdns
> > at all.
> >
> > I do whitelist all mailling lists as well, they never see SA.
> >
> > In my position, Botnet is good. But if I were an ISP I could not use it.
> > Impossible. Totally impossible.
>
> You never tried, nor need to, and say it is impossible? Not true (have you
> heard of the trusted_networks setting), it is possible and any ISP who uses SA
> would gain by using it.
>
> The work Botnet does is similar to graylists, a good one stops suspicious mail
> servers for a while, if they insist they'll pass the graylist and get scored by
> Botnet, how much you score them is your choice.
Well, like I said, we had big problems using anything in Botnet except
nordns. Does anyone have a good words list I could try? I have set
BOTNET_CLIENT to 1.0 and that seems to start killing these messages. I
also have everything else set to 0 except BOTNET_NORDNS at 4.5. Does all
the other settings being zero effect my BOTNET_CLIENT scores or will it
continue to calculate the BOTNET_CLIENTWORDS, etc, as part of
BOTNET_CLIENT?
--
Robert
Re: Suggested botnet rule scores
Posted by René Berber <r....@computer.org>.
Jari Fredriksson wrote:
> Botnet is bad AFAIK bad for anyone running an ISP or so.
>
> I'm a lone one and I know that nobody sending me email is not using a Linux
> box with his own server, so I can drop all mail from dynamic dns or no rdns
> at all.
>
> I do whitelist all mailling lists as well, they never see SA.
>
> In my position, Botnet is good. But if I were an ISP I could not use it.
> Impossible. Totally impossible.
You never tried, nor need to, and say it is impossible? Not true (have you
heard of the trusted_networks setting), it is possible and any ISP who uses SA
would gain by using it.
The work Botnet does is similar to graylists, a good one stops suspicious mail
servers for a while, if they insist they'll pass the graylist and get scored by
Botnet, how much you score them is your choice.
--
René Berber
Re: Suggested botnet rule scores
Posted by Jari Fredriksson <ja...@iki.fi>.
Botnet is bad AFAIK bad for anyone running an ISP or so.
I'm a lone one and I know that nobody sending me email is not using a Linux box with his own server, so I can drop all mail from dynamic dns or no rdns at all.
I do whitelist all mailling lists as well, they never see SA.
In my position, Botnet is good. But if I were an ISP I could not use it. Impossible. Totally impossible.
Re: Suggested botnet rule scores
Posted by Kai Schaetzl <ma...@conactive.com>.
Robert Fitzpatrick wrote on Fri, 17 Aug 2007 11:23:56 -0400:
> Still no good, I only get the message, no debug info...:(
But you get it on the screen, right? You may have to redirect std:err or
what it's called as well to get the dbg output in that file.
> Anyone can tell us what these scores do and/or how called?
detailed explanation of which column is for which mode is in the
documentation and/or on the wiki.
Kai
--
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
RE: Suggested botnet rule scores
Posted by tkb2766 <tk...@anroet.com>.
> -----Original Message-----
> From: Robert Fitzpatrick [mailto:lists@webtent.net]
> Sent: Saturday, 18 August 2007 1:24
> To: users@spamassassin.apache.org
> Subject: Re: Suggested botnet rule scores
>
> On Fri, 2007-08-17 at 16:31 +0200, Kai Schaetzl wrote:
> > Robert Fitzpatrick wrote on Fri, 17 Aug 2007 08:46:25 -0400:
> >
> > > I tried 'spamassassin -D > results.txt <
> > > myspamfile', but only gives me the results of the tests.
> >
> > spamassassin -D <myspamfile >results.txt
> >
> > should do it.
>
> Still no good, I only get the message, no debug info...:(
To get the debug info into a file, you need to send std_err output to file
as debug output is really error output. You have 2 options:
option 1 - pipe to less:
spamassassin --lint -D < spamfile 2>&1 | less
option 2 - redirect to text file:
spamassassin --lint -D < spamfile 2>&1 > file.txt
That ought to do it!
Cheers,
tkb.
Re: John Nicolau spam
Posted by Loren Wilton <lw...@earthlink.net>.
> That's exactly what I did, but it is not working because they keep on
> coming
> in.
Perhaps you should show us the rule. I see no reason the following wouldn't
work:
body Nicolau /\bNicolau\b/i
score Nicolau 5
Or:
header F_Nicolau From =~ /Nicolau/i
body B_Nicolau /\bNicolau\b/i
meta Yup_Nicolau F_Nicolau && B_Nicolau
score Yup_Nicolau 25
Loren
RE: John Nicolau spam
Posted by Leonardo Magallon <lm...@itsinc.com>.
That's exactly what I did, but it is not working because they keep on coming
in.
-----Original Message-----
From: Jerry Durand [mailto:jdurand@interstellar.com]
Sent: Friday, August 17, 2007 11:38 AM
To: Leonardo Magallon
Cc: users@spamassassin.apache.org
Subject: Re: John Nicolau spam
At 09:34 AM 8/17/2007, Leonardo Magallon wrote:
> Has anyone else been experiencing the same problem?
We get Robyn Miller mail, a year or so ago I added a filter to
local.cf that adds some huge score to any male with that string in it.
One just came in today as "Miller Robyn", I may have to update my filter.
--
Jerry Durand, Durand Interstellar, Inc. www.interstellar.com
tel: +1 408 356-3886, USA toll free: 1 866 356-3886
Skype: jerrydurand
Re: John Nicolau spam
Posted by Jerry Durand <jd...@interstellar.com>.
At 09:34 AM 8/17/2007, Leonardo Magallon wrote:
> Has anyone else been experiencing the same problem?
We get Robyn Miller mail, a year or so ago I added a filter to
local.cf that adds some huge score to any male with that string in it.
One just came in today as "Miller Robyn", I may have to update my filter.
--
Jerry Durand, Durand Interstellar, Inc. www.interstellar.com
tel: +1 408 356-3886, USA toll free: 1 866 356-3886
Skype: jerrydurand
John Nicolau spam
Posted by Leonardo Magallon <lm...@itsinc.com>.
I've had it with this emails coming from John Nicolau and with that name in
the email body. They started coming in since the beginning of this week.
I added a rule that basically says that if the word "Nicolau" is in the
body, to refuse the email but it is apparently not working.
Has anyone else been experiencing the same problem?
Thanks.
Re: Suggested botnet rule scores
Posted by Robert Fitzpatrick <li...@webtent.net>.
On Fri, 2007-08-17 at 16:31 +0200, Kai Schaetzl wrote:
> Robert Fitzpatrick wrote on Fri, 17 Aug 2007 08:46:25 -0400:
>
> > I tried 'spamassassin -D > results.txt <
> > myspamfile', but only gives me the results of the tests.
>
> spamassassin -D <myspamfile >results.txt
>
> should do it.
Still no good, I only get the message, no debug info...:(
>
> 50_scores.cf:score ACT_NOW_CAPS 0.948 0.001 1.259 0.792
>
> That might explain it. The second score is used on your setup. Don't
> remember which column is for what. Is this with network tests on?
>
We use amavisd-new and in the conf file $sa_local_tests_only = 0;
Anyone can tell us what these scores do and/or how called? This does
look like I'm hitting the second score. Thanks :)
--
Robert
Re: Suggested botnet rule scores
Posted by Kai Schaetzl <ma...@conactive.com>.
Robert Fitzpatrick wrote on Fri, 17 Aug 2007 08:46:25 -0400:
> I tried 'spamassassin -D > results.txt <
> myspamfile', but only gives me the results of the tests.
spamassassin -D <myspamfile >results.txt
should do it.
50_scores.cf:score ACT_NOW_CAPS 0.948 0.001 1.259 0.792
That might explain it. The second score is used on your setup. Don't
remember which column is for what. Is this with network tests on?
Kai
--
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
Re: Suggested botnet rule scores
Posted by Robert Fitzpatrick <li...@webtent.net>.
On Fri, 2007-08-17 at 00:31 +0200, Kai Schaetzl wrote:
> It seems you lowered the score of ACT_NOW_CAPS. If you have done this
> with
> a lot of rules, it's understandable that they don't help ;-)
Good eyes, I didn't even see that. I have checked my local.cf, where is
the only place I lower or alter scores in any way, and ACT_NOW_CAPS is
not in there. Trying to track down why this is coming back zero, how can
I grep the debug output of spamassassin? Is there a way to get the debug
info into a file for searching? I tried 'spamassassin -D > results.txt <
myspamfile', but only gives me the results of the tests.
--
Robert
Re: Suggested botnet rule scores
Posted by Kai Schaetzl <ma...@conactive.com>.
Robert Fitzpatrick wrote on Thu, 16 Aug 2007 16:15:24 -0400:
> Wondering what score
> settings others are using for Botnet or are you able to kill these
> messages without it?
No, this message has too few generic spam signs. But if you get a lot of
them you can easily take out some of the typical words and create rules.
As they do not obfuscate anything that should be easy.
Here's what current unchanged BotNet scores.
5.0 BOTNET Relay might be a spambot or virusbot
[botnet0.8,ip=72.51.59.60,rdns=60.bo.static.symmetrixns1.com,maildomain=si
tores.villanously.com,client,clientwords]
That would have pumped it over the threshold, so, it was right in this
case. But I'm not sure what this decision based on and if I like it and
this high score. (Just started evaluating BotNet a few days ago, but since
I actually don#t get much BotNet spam I can't really evaluate it ;-) The
rdns has "static" in it. I don't see how this can point to a Botnet
without knowing it is one, those many dots?
It seems you lowered the score of ACT_NOW_CAPS. If you have done this with
a lot of rules, it's understandable that they don't help ;-)
Kai
--
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com