You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Dr Robert Young <rc...@aliconsultants.com> on 2005/07/13 18:11:17 UTC
very "low" scoring (ie large negative numbers) spam
I have several spam that scored very low on SA 3.0.4 w/ milter-spamc
0.25
X-Spam-Status: NO, hits=-145.70 required=5.00
X-Spam-Status: NO, hits=-153.70 required=5.00
X-Spam-Status: NO, hits=-146.00 required=5.00
X-Spam-Status: NO, hits=-153.80 required=5.00
X-Spam-Status: NO, hits=-153.90 required=5.00
Since all the scores are similar, I am wondering if there is a common
"cause" .? Are there any suggestions on how to investigate the source
of the wildly "negative" score, or does it look familiar?
Is there an option in spamd/milter-spamc to allow the "detailed"
results of spam scoring to be returned (which test contributed which
score to the total) for ALL email ( spam and non-spam)??
________________________________________________________
Dr. Robert Young
ALI Database Consultants
1151 Williams Dr
Aiken SC 29803
USA
WWW: http://www.aliconsultants.com
Tele: 1-803-648-5931
Toll free in US: 1-866-257-8970 Fax:1-803-641-0345
Email: rcyoung@aliconsultants.com
"Source of Rdb Controller, software for database analysis &
performance tuning"
Re: very "low" scoring (ie large negative numbers) spam
Posted by Theo Van Dinter <fe...@apache.org>.
On Wed, Jul 13, 2005 at 03:56:50PM -0700, Loren Wilton wrote:
> Another possibility is they are hitting TWO whitelist rules and
> getting -200, and then getting 50..60 points added back to the score from
> other rule hits.
That's not really possible. In 3.0, there's only a single -100 whitelist
rule. It'll only ever hit once.
I'm voting for AWL, probably with a static whitelist as well.
--
Randomly Generated Tagline:
Oh, I love your magazine. My favorite section is `How to Increase
Your Word Power.' That thing is really, really, really ... good.
-- Homer Simpson
Mr. Lisa Goes To Washington
Re: very "low" scoring (ie large negative numbers) spam
Posted by Loren Wilton <lw...@earthlink.net>.
> I have several spam that scored very low on SA 3.0.4 w/ milter-spamc 0.25
>
> X-Spam-Status: NO, hits=-145.70 required=5.00
> X-Spam-Status: NO, hits=-153.70 required=5.00
> X-Spam-Status: NO, hits=-146.00 required=5.00
> X-Spam-Status: NO, hits=-153.80 required=5.00
> X-Spam-Status: NO, hits=-153.90 required=5.00
>
> Since all the scores are similar, I am wondering if there is a common
> "cause" .
Almost certainly.
> Are > there any suggestions on how to investigate the source
> of the wildly "negative" score, or does it look familiar?
A score of -100 almost certainly indicates a whitelist hit of some sort or
other.
The 40..50 part of the score is harder to determine. I would guess these
are similar mails and are accumulating about the same hits, which could be
any old batch of rules.
Another possibility is they are hitting TWO whitelist rules and
getting -200, and then getting 50..60 points added back to the score from
other rule hits.
If you look at one of those messages (if you have it) it should show the
rules hit, even without the scores. Many of us could fairly quickly tell
you where the scores are coming from.
Loren
Re: very "low" scoring (ie large negative numbers) spam
Posted by Matt Kettler <mk...@evi-inc.com>.
Dr Robert Young wrote:
> I have several spam that scored very low on SA 3.0.4 w/ milter-spamc 0.25
>
> *X-Spam-Status: * NO, hits=-145.70 required=5.00
> *X-Spam-Status: * NO, hits=-153.70 required=5.00
> *X-Spam-Status: * NO, hits=-146.00 required=5.00
> *X-Spam-Status: * NO, hits=-153.80 required=5.00
> X-Spam-Status: NO, hits=-153.90 required=5.00
>
>
> Since all the scores are similar, I am wondering if there is a common
> "cause" .? Are there any suggestions on how to investigate the source of
> the wildly "negative" score, or does it look familiar?
Looks like the messages are matching a whitelist_from or all_spam_to statement.
Commonly this is somebody doing the "simple" thing and adding:
whitelist_from *@mydomain.com
Which unfortunately whitelists all spam that forges itself as being from an
address in your domain.
You'll want to use whitelist_from_rcvd instead, or better yet, modify your setup
so internal mail doesn't get fed to SA at all and remove the whitelist.
Re: Proper way to override scores
Posted by jdow <jd...@earthlink.net>.
----- Original Message -----
From: "Debbie D" <we...@beautytech.com>
To: <us...@spamassassin.apache.org>
Sent: 2005 July, 14, Thursday 06:20
Subject: Re: Proper way to override scores
> I didn't cross post that I am aware of.. I know it is poor form!!
According
> to my SENT box it went to: gmane.mail.spam.spamassassin.general only
>
>
>
> "jdow" <jd...@earthlink.net> wrote in message
> news:00dd01c58840$583797d0$1225a8c0@kittycat...
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Somehow that got poked in there. And that looks like it is an access to
this list via news protocols. You must have used an archive site that used
a strange means to post to the list. The usual address is:
users@spamassassin.apache.org. And it's curious that this is indeed where
the mail was directed.
Ah - there's the culprit: Sender: news <ne...@sea.gmane.org>
So my email program thought of it as a news posting.
Of course, you MIGHT want to join the list. Email to:
mailto:users-subscribe@spamassassin.apache.org should in theory start
the subscription cycle.
{^_^}
Re: Proper way to override scores
Posted by Debbie D <we...@beautytech.com>.
I didn't cross post that I am aware of.. I know it is poor form!! According
to my SENT box it went to: gmane.mail.spam.spamassassin.general only
"jdow" <jd...@earthlink.net> wrote in message
news:00dd01c58840$583797d0$1225a8c0@kittycat...
> From: "Debbie D" <we...@beautytech.com>
>
>> I often want to alter the scores of already set filters in the SARE and
>> other custom filter sets.. what/where is the proper places to do this
>> without altering each individual set which will get over-written down the
>> road
>
> By the way, Debbie, it is poor form to multiply crosspost newsgroups
> and mailing lists. This particular mailing list is perhaps your best
> shot for solid knowledge. The GURUs are here.
>
> {^_-}
>
>
>
Re: Proper way to override scores
Posted by jdow <jd...@earthlink.net>.
From: "Debbie D" <we...@beautytech.com>
> I often want to alter the scores of already set filters in the SARE and
> other custom filter sets.. what/where is the proper places to do this
> without altering each individual set which will get over-written down the
> road
By the way, Debbie, it is poor form to multiply crosspost newsgroups
and mailing lists. This particular mailing list is perhaps your best
shot for solid knowledge. The GURUs are here.
{^_-}
Re: Proper way to override scores
Posted by Matt Kettler <mk...@evi-inc.com>.
Debbie D wrote:
> I often want to alter the scores of already set filters in the SARE and
> other custom filter sets.. what/where is the proper places to do this
> without altering each individual set which will get over-written down the
> road
>
> thanks
>
>
>
You want to add them to a .cf file in /etc/mail/spamassassin, alongside all your
other custom rulesets. (If your custom rulesets are in /usr/share/spamassassin/,
move them ONLY the default rules should be there. This directory WILL be
obliterated on upgrade.)
You also want your score-over-ride to be parsed AFTER the SARE rule files are
parsed. In SA, the last-file read has the final say in what the score of a rule
is, and it's done that way intentionally to facilitate over-riding scores.
SA parses all the *.cf files in /etc/mail/spamassassin in alphabetical order, so
z_score_override.cf might be a good choice of filename.
Re: Proper way to override scores
Posted by Debbie D <we...@beautytech.com>.
"Kai Schaetzl" <ma...@conactive.com> wrote in message
news:VA.00001dad.011f98a7@virtual-access.org...
> Debbie D wrote on Thu, 14 Jul 2005 09:26:09 -0400:
>
>> I simply add to my custom or create a new custom lets say called:
>> 10_custom_scores.cf
>
> the name is not important, it just needs to be in /etc/mail/spamassassin
> if this is your local config dir.
>
>>
>> score SARE_ADLTSUB6 8.500 # type=obfu
>>
>> is that correct?? Just a list of the altered scores??
>
> Yes. But you don't need the comment. And if you want to disable a rule
> just set it to 0.
Kai.. I just copy/pasted that score line.. I realize I don't need the
comment.. and yep I know it belongs in /etc/mail/spamassassin -- and I also
know I need to --lint & restart EXIM to make it see the rules :)
thanks that's exactly what I wanted to know..
Re: Proper way to override scores
Posted by Kai Schaetzl <ma...@conactive.com>.
Debbie D wrote on Thu, 14 Jul 2005 09:26:09 -0400:
> I simply add to my custom or create a new custom lets say called:
> 10_custom_scores.cf
the name is not important, it just needs to be in /etc/mail/spamassassin
if this is your local config dir.
>
> score SARE_ADLTSUB6 8.500 # type=obfu
>
> is that correct?? Just a list of the altered scores??
Yes. But you don't need the comment. And if you want to disable a rule
just set it to 0.
Kai
--
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
IE-Center: http://ie5.de & http://msie.winware.org
Re: Proper way to override scores
Posted by Loren Wilton <lw...@earthlink.net>.
> Thanks.. I just wanted to also verify the format of the over rides..
The score override is exactly the same format as the original score line.
The word "score", the identical rule name, and the new score or scores.
> For instance.. if this is the rule I want to override:
>
> header SARE_ADLTSUB6 Subject =~ /(?!\bfuck)\bf.?u.?c.?k/i
> describe SARE_ADLTSUB6 Apparent spam seems to contain porn subject
> score SARE_ADLTSUB6 3.500 # type=obfu
>
> I simply add to my custom or create a new custom lets say called:
> 10_custom_scores.cf
Bad choice.
> score SARE_ADLTSUB6 8.500 # type=obfu
>
> is that correct?? Just a list of the altered scores??
Yes, simply put a new score on. HOwever assuming you use the normal 5.0
threshold, an 8.5 score would be a bit, um, drastic. Maybe in your case
(I'm assuming a publich gradeschool) that would be justified; but it needs
some careful thought. In general the SA philosophy is to NOT have any
single rule that can by itself mark a mail as spam, as it leads far too
easily to getting FPs.
> >From your response I take it that SA will assign the highest found
score??
Nope.
> Not: run thru the sets in order assigning the last found score?
Yep. Which is why your suggested file name is a bad choice. "10_" will be
parsed very early, and "70_" will then override it.
A better choice would be "99_score_overrides.cf". It isn't tto likely that
something else will override that file.
BTW, you can do this with 'description' lines also if you need to.
Loren
Re: Proper way to override scores
Posted by jdow <jd...@earthlink.net>.
From: "Bret Miller" <br...@wcg.org>
> > Thanks.. I just wanted to also verify the format of the over rides..
> >
> > For instance.. if this is the rule I want to override:
> >
> > header SARE_ADLTSUB6 Subject =~ /(?!\bfuck)\bf.?u.?c.?k/i
> > describe SARE_ADLTSUB6 Apparent spam seems to contain porn subject
> > score SARE_ADLTSUB6 3.500 # type=obfu
> >
> >
> > I simply add to my custom or create a new custom lets say called:
> > 10_custom_scores.cf
> >
> > score SARE_ADLTSUB6 8.500 # type=obfu
> >
> > is that correct?? Just a list of the altered scores??
> >
> > >From your response I take it that SA will assign the highest
> > found score??
> > Not: run thru the sets in order assigning the last found score?
>
> No, SA assigns the LAST found score. That's why Joanne suggested calling
> it zzz-myscores.cf so it would sort LAST in the folder.
I'm merely suggesting placing it in the same folder as local.cf rather
than with the spamassassin distributed scores and such. According to
documentation it appears that the /etc/share/spamassassin directory MAY
get deleted and replaced by a new version of spamassassin when it is
installed. The /etc/mail/spamassassin directory is left along if it
exists. (So you have to revise local.cf for some changes on some upgrades.)
The /etc/mail/spamassassin directory contents are handled after the
stock rules are parsed so they override them no matter what their name
is. However for scores I might be inclined to use a name that sorts
after some of the SARE 99_ rules.
{^_^}
RE: Proper way to override scores
Posted by Bret Miller <br...@wcg.org>.
> Thanks.. I just wanted to also verify the format of the over rides..
>
> For instance.. if this is the rule I want to override:
>
> header SARE_ADLTSUB6 Subject =~ /(?!\bfuck)\bf.?u.?c.?k/i
> describe SARE_ADLTSUB6 Apparent spam seems to contain porn subject
> score SARE_ADLTSUB6 3.500 # type=obfu
>
>
> I simply add to my custom or create a new custom lets say called:
> 10_custom_scores.cf
>
> score SARE_ADLTSUB6 8.500 # type=obfu
>
> is that correct?? Just a list of the altered scores??
>
> >From your response I take it that SA will assign the highest
> found score??
> Not: run thru the sets in order assigning the last found score?
No, SA assigns the LAST found score. That's why Joanne suggested calling
it zzz-myscores.cf so it would sort LAST in the folder.
Bret
Re: Proper way to override scores
Posted by Debbie D <we...@beautytech.com>.
Herb,
Thanks.. I just wanted to also verify the format of the over rides..
For instance.. if this is the rule I want to override:
header SARE_ADLTSUB6 Subject =~ /(?!\bfuck)\bf.?u.?c.?k/i
describe SARE_ADLTSUB6 Apparent spam seems to contain porn subject
score SARE_ADLTSUB6 3.500 # type=obfu
I simply add to my custom or create a new custom lets say called:
10_custom_scores.cf
score SARE_ADLTSUB6 8.500 # type=obfu
is that correct?? Just a list of the altered scores??
>From your response I take it that SA will assign the highest found score??
Not: run thru the sets in order assigning the last found score?
Thanks
"jdow" <jd...@earthlink.net> wrote in message
news:00d801c58840$03006850$1225a8c0@kittycat...
> From: "Herb Martin" <He...@learnquick.com>
>
>> > -----Original Message-----
>> > From: news [mailto:news@sea.gmane.org] On Behalf Of Debbie D
>> >
>> > I often want to alter the scores of already set filters in
>> > the SARE and other custom filter sets.. what/where is the
>> > proper places to do this without altering each individual set
>> > which will get over-written down the road
>>
>> Two good ways (there may be others):
>>
>> Put the overrides in your local.cf
>>
>> OR
>>
>> Put them in a .cf file in the same directory with,
>> and with a later alphabetical listing, than the other
>> SARE, etc files so that the scores must follow the setup
>> and defaults.
>> For me this is /usr/share/spamassassin
>> ...and zzz-myscores.cf should work well.
>>
>> You could also use different file for each "SARE" or other
>> custom file, just ensure it gets a "higher" (later) alphabetical
>> name, so 70_sare_unsub.cf could be rescored in
>> 70_sare_unsub_scores.cf or just 71_sare_unsub.cf
>>
>> The main (included) spamassassin scores are in the 50xxxx.cf
>> file so that is follows after the 10-40xxx default test files.
>
> No to both of them, Herb.
>
> Place them into the /etc/mail/spamassassin (or /etc/spamassassin depending
> on where the local.cf file is.) Make a new file and put them into that
> file. It's cleaner than getting local.cf all cluttered.
>
> Never place new scores or rules into the /usr/share/spamassassin
> directory. They WILL get deleted or replaced when you update. You are
> better off to simply override the /usr/share/spamassassin scores in a
> new rules file in the same directory with the local.cf file. (That is
> also where you want to install your SARE rules. And you DO want a
> bundle of them, likely as not. (The set I use dramatically improve
> the falses from a bare fresh install.) These are found at several
> locations listed in the spamassassin wiki. The basic SARE rules site
> is http://www.rulesemporium.com/. http://www.exit0.us is another good
> site to visit for rules. http://www.rulesemporium.com/links.htm has some
> nice HOW-TO links that will help.
>
> {^_^} Joanne
>
>
Re[2]: Proper way to override scores
Posted by Robert Menschel <Ro...@Menschel.net>.
Hello Herb,
Thursday, July 14, 2005, 12:12:26 AM, you wrote:
>> Never place new scores or rules into the
>> /usr/share/spamassassin directory. They WILL get deleted or
>> replaced when you update.
HM> Really? I didn't know that (obviously) although I have updated
HM> several (minor) versions of SpamAssassin and never lost any
HM> added files in the share directory.
HM> Of course one must no modify the default files, or create one
HM> with a name likely to be used in a new version but new files
HM> seem to survive upgrades just fine.
HM> Are you sure these will get overwritten or deleted?
Yes, eventually, depending on how you update/install.
Some package updates will delete the directory and start over. I
believe a "make clean" will do so as well. You may continue to be
lucky for several minor releases, but you should count on
/usr/share/spamassassin eventually being wiped and replaced.
Bob Menschel
RE: Proper way to override scores
Posted by Herb Martin <He...@learnquick.com>.
Joanne wrote:
> No to both of them, Herb.
>
> Place them into the /etc/mail/spamassassin (or
> /etc/spamassassin depending on where the local.cf file is.)
> Make a new file and put them into that file. It's cleaner
> than getting local.cf all cluttered.
That seems a good variation on the other two methods.
> Never place new scores or rules into the
> /usr/share/spamassassin directory. They WILL get deleted or
> replaced when you update.
Really? I didn't know that (obviously) although I have updated
several (minor) versions of SpamAssassin and never lost any
added files in the share directory.
Of course one must no modify the default files, or create one
with a name likely to be used in a new version but new files
seem to survive upgrades just fine.
Are you sure these will get overwritten or deleted?
> You are better off to simply
> override the /usr/share/spamassassin scores in a new rules
> file in the same directory with the local.cf file. (That is
> also where you want to install your SARE rules. And you DO
> want a bundle of them, likely as not. (The set I use
> dramatically improve the falses from a bare fresh install.)
It still sounds like good advice (yours that is.)
> These are found at several locations listed in the
> spamassassin wiki. The basic SARE rules site is
> http://www.rulesemporium.com/. http://www.exit0.us is another
> good site to visit for rules.
> http://www.rulesemporium.com/links.htm has some nice HOW-TO
> links that will help.
>
> {^_^} Joanne
>
Re: Proper way to override scores
Posted by jdow <jd...@earthlink.net>.
From: "Herb Martin" <He...@learnquick.com>
> > -----Original Message-----
> > From: news [mailto:news@sea.gmane.org] On Behalf Of Debbie D
> >
> > I often want to alter the scores of already set filters in
> > the SARE and other custom filter sets.. what/where is the
> > proper places to do this without altering each individual set
> > which will get over-written down the road
>
> Two good ways (there may be others):
>
> Put the overrides in your local.cf
>
> OR
>
> Put them in a .cf file in the same directory with,
> and with a later alphabetical listing, than the other
> SARE, etc files so that the scores must follow the setup
> and defaults.
> For me this is /usr/share/spamassassin
> ...and zzz-myscores.cf should work well.
>
> You could also use different file for each "SARE" or other
> custom file, just ensure it gets a "higher" (later) alphabetical
> name, so 70_sare_unsub.cf could be rescored in
> 70_sare_unsub_scores.cf or just 71_sare_unsub.cf
>
> The main (included) spamassassin scores are in the 50xxxx.cf
> file so that is follows after the 10-40xxx default test files.
No to both of them, Herb.
Place them into the /etc/mail/spamassassin (or /etc/spamassassin depending
on where the local.cf file is.) Make a new file and put them into that
file. It's cleaner than getting local.cf all cluttered.
Never place new scores or rules into the /usr/share/spamassassin
directory. They WILL get deleted or replaced when you update. You are
better off to simply override the /usr/share/spamassassin scores in a
new rules file in the same directory with the local.cf file. (That is
also where you want to install your SARE rules. And you DO want a
bundle of them, likely as not. (The set I use dramatically improve
the falses from a bare fresh install.) These are found at several
locations listed in the spamassassin wiki. The basic SARE rules site
is http://www.rulesemporium.com/. http://www.exit0.us is another good
site to visit for rules. http://www.rulesemporium.com/links.htm has some
nice HOW-TO links that will help.
{^_^} Joanne
RE: Proper way to override scores
Posted by Herb Martin <He...@learnquick.com>.
> -----Original Message-----
> From: news [mailto:news@sea.gmane.org] On Behalf Of Debbie D
>
> I often want to alter the scores of already set filters in
> the SARE and other custom filter sets.. what/where is the
> proper places to do this without altering each individual set
> which will get over-written down the road
Two good ways (there may be others):
Put the overrides in your local.cf
OR
Put them in a .cf file in the same directory with,
and with a later alphabetical listing, than the other
SARE, etc files so that the scores must follow the setup
and defaults.
For me this is /usr/share/spamassassin
...and zzz-myscores.cf should work well.
You could also use different file for each "SARE" or other
custom file, just ensure it gets a "higher" (later) alphabetical
name, so 70_sare_unsub.cf could be rescored in
70_sare_unsub_scores.cf or just 71_sare_unsub.cf
The main (included) spamassassin scores are in the 50xxxx.cf
file so that is follows after the 10-40xxx default test files.
--
Herb Martin
Proper way to override scores
Posted by Debbie D <we...@beautytech.com>.
I often want to alter the scores of already set filters in the SARE and
other custom filter sets.. what/where is the proper places to do this
without altering each individual set which will get over-written down the
road
thanks
Re: very "low" scoring (ie large negative numbers) spam
Posted by Andy Jezierski <aj...@stepan.com>.
Dr Robert Young <rc...@aliconsultants.com> wrote on 07/13/2005 11:11:17
AM:
> I have several spam that scored very low on SA 3.0.4 w/ milter-spamc
> 0.25
>
> X-Spam-Status: NO, hits=-145.70 required=5.00
> X-Spam-Status: NO, hits=-153.70 required=5.00
> X-Spam-Status: NO, hits=-146.00 required=5.00
> X-Spam-Status: NO, hits=-153.80 required=5.00
> X-Spam-Status: NO, hits=-153.90 required=5.00
>
>
> Since all the scores are similar, I am wondering if there is a common
> "cause" .? Are there any suggestions on how to investigate the source
> of the wildly "negative" score, or does it look familiar?
>
> Is there an option in spamd/milter-spamc to allow the "detailed"
> results of spam scoring to be returned (which test contributed which
> score to the total) for ALL email ( spam and non-spam)??
You're more than likely hitting a whitelist entry in your local.cf
Add a -A to your milter startup for a detailed report.
Andy