You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Yassen Damyanov <yd...@troyer.co.at> on 2004/12/10 15:47:51 UTC
Sudden and strange SA Score reduction
Hi SA User List,
Here's my case: postfix + amavisd-new + SpamAssassin 2.64
working on a Gentoo Linux box, serving as a mail server for
serveral virtual domains.
Some SpamAssassin details: Bayes learning activated recently,
based on about 300 spam mails and 200 ham mails, which accumulate
in IMAP folders and are scanned using sa-learn via cron job three
times a day.
It all seems to work, but I see SA passing through some obvious spam,
so I decided to look. And what I dicovered was very surprising for me:
SA computes the score well, but suddenly lowers it significantly
exactly before returning an answer to amavisd.
Here an examples:
.
.
debug: running raw-body-text per-line regexp tests; score so far=4.166
debug: running uri tests; score so far=4.166
debug: uri tests: Done uriRE
debug: running full-text regexp tests; score so far=4.166
debug: all '*From' addrs: fmyldsyx@AMarseille-207-1-4-127.w81-249.abo.wanadoo.fr
debug: all '*To' addrs: test@jens-weigel.de kht@troyer.co.at sgospodinova@troyer.co.at vtodorov@troyer.co.at yd@troyer.co.at ydamian
ov@mail2.troyer.co.at
debug: forged-HELO: from=media-c.local helo=troyer.co.at by=media-c.de
debug: forged-HELO: mismatch on HELO: 'troyer.co.at' != 'media-c.local'
debug: forged-HELO: from=wanadoo.fr helo= by=troyer.co.at
debug: forged-HELO: mismatch on from: 'media-c.local' != 'troyer.co.at'
debug: running meta tests; score so far=5.53
debug: auto-learn? ham=0.2, spam=8, body-hits=4.166, head-hits=1.364
debug: auto-learn: currently using scoreset 2. recomputing score based on scoreset 0.
debug: Score set 0 chosen.
debug: auto-learn: original score: 5.53, recomputed score: 4.922
debug: Score set 2 chosen.
debug: auto-learn? no: inside auto-learn thresholds
debug: is spam? score=0.629 required=6.8 tests=BAYES_00,DATE_IN_PAST_12_24,SARE_ADULT2,SARE_OBFUPORNO
Then the message is tagged "X-Spam-Status: No, hits=0.6".
This is an obvious adult site adv. and SARE rules do a good job. But why the score
is lowered at the end?
If I use spamassassin command line tool for the SAME message, I get very good
result:
# spamassassin < 1102684670.M621242P31174V0000000000006210I00006396_0.prodo,S=1996:2,S
X-Spam-Checker-Version: SpamAssassin 2.64 (2004-01-11) on prodo.media-c.local
X-Spam-Level: ******
X-Spam-Status: No, hits=6.5 required=6.8 tests=BEST_PORN,DATE_IN_PAST_12_24,
SARE_ADULT2,SARE_OBFUPORNO autolearn=no version=2.64
-- cut here ------------------------
As is not difficult to gues, I would like to have THESE scores, non-lowered!
I suspected the bayesian learning to be blamed... but when checking the learning
sesssions logs, everyhting is correct, spam and ham are perfectly sorted and
learning is conducted as appropriate. So I am stuck.
Why does this mailfunction appear? Suggestions for fixing?
Any help would greatly appreciated.
Yassen
P.S. Please find below enclosed some configs as /etc/mail/spamassassin/local.cf,
and others.
--
Yassen Damyanov
phone: +359-32-968-903
email: yd@troyer.co.at
ICQ# : 169382108
web : www.troyer-is.com
# cat /etc/mail/spamassassin/local.cf | sed -n -e '/^[\t ]*#\|^[\t ]*$/!p'
trusted_networks 127.0.0.
required_hits 6.8
rewrite_subject 1
subject_tag [SPAM?]
report_safe 1
use_terse_report 0
use_bayes 1
auto_learn 1
skip_rbl_checks 0
use_razor2 1
use_dcc 1
use_pyzor 1
ok_languages de en
ok_locales de en
# cat /etc/amavisd.conf | sed -n -e '/^[\t ]*#\|^[\t ]*$/!p'
# -- Relevant Sections Only --
$MYHOME = '/var/run/amavis'; # (default is '/var/amavis')
$spam_quarantine_to = new_RE(
[qr'^([^@]*)@([^@]*)$'i => '/var/virtual/hosts/${2}/home/${1}/.maildir/.Junk/new'],
[qr/.*/ => 'spam-quarantine'] );
$X_HEADER_TAG = 'X-Virus-Scanned';
$X_HEADER_LINE = "by amavisd-new-20030616:clamav-0.80 at prodo.media-c.de";
$undecipherable_subject_tag = '[UNCHECKED] ';
$remove_existing_x_scanned_headers = 0;
$remove_existing_spam_headers = 0;
$sa_local_tests_only = 1;
$sa_timeout = 30;
$sa_mail_body_size_limit = 150*1024;
$sa_tag_level_deflt = -99.0;
$sa_tag2_level_deflt = 4.2;
$sa_kill_level_deflt = 5.8;
$sa_dsn_cutoff_level = 10.0;
$sa_spam_subject_tag = '[SPAM?] ';
$sa_debug = 1;
When running sa-learning, I use
Re: Sudden and strange SA Score reduction
Posted by Kris Deugau <kd...@vianet.ca>.
Yassen Damyanov wrote:
> On Friday 10 December 2004 18:21, Kris Deugau wrote:
> > I've
> > had trouble in the past with Bayes learning very low-scoring spam
> > as ham - so I lowered the autolearn-as-ham threshold to -0.1.
> I came to a conclusion that some real spams got scored very low and
> poisoned the Bayes db, so it started to make mistakes and thus
> poisoned itself even more, then made worse mistakes, ... etc.
Yep. For a while about a year ago I was seeing spams coming in that
were only hitting a rule called HTML_ONLY (IIRC). This rule scored
0.1. My autolearn threshold at the time was the default 0.2, and so
these spams were getting learned as ham.
I dropped the autolearn threshold, and I've been fortunate to have ~2%
of my users regularly forward FNs to me (properly, no less!) for manual
learning so I've been able to feed quite a few such messages back in as
spam. (Learning a message as spam when it's previously been learned as
ham will effectively result in the Bayes db appearing as if that message
had never been learned as ham - in theory, and so far it's worked for
me.)
> I switched off the auto_learning (not needed IMHO when we have
> regular manual learning sessions) and then deleted the old database
> and rerun the manual learning script.
Mmmh. I've left autolearn enabled in order to leave me with less work.
You *do* have to make sure you get feedback, but if you have a regular
process in place for learning a block of legit mail and a block of spam
(confirmed by hand-sorting) then it may not be quite as important.
I think the biggest problem many people have with Bayes is the early
maintenance - if you make a few tweaks (like lowering the
autolearn-as-ham threshold) and watch it closely for two to three weeks,
you should end up with a pretty good start.
I've *never* had to wipe the global Bayes files on the two systems I've
been running SA on at work; they've been running Bayes since shortly
after 2.54 was released. (I waited a little to upgrade from 2.44 on
these machines, 2.50 thru 2.53 ended up being a little unstable and
flaky IIRC.)
I've never had to wipe the Bayes files on my personal email on my own
server. *BUT*.... In all three cases I've watched my email (or
customer feedback regarding their email) and I've looked ahead a little
to try to forsee problems and fix them before they become BIG problems.
> BTW, how to interpret things like "tests=BAYES_56" or
> "tests=BAYES_00" in the X-Spam-Status header?
That's just the list of tests that triggered or matched, out of the
complete list of tests SA runs. Depending on your setup, the
appropriate score for that test is added to the total.
The BAYES_* tests are a probability check - based on the tokens in the
message compared with the tokens in the DB, the message is considered to
have a probability n% of being spam. There are sets of Bayes scores for
different probability ranges; see the SA docs and default rule files to
see exactly where the boundaries are.
-kgd
--
Get your mouse off of there! You don't know where that email has been!
Re: Sudden and strange SA Score reduction
Posted by Yassen Damyanov <yd...@troyer.co.at>.
Thanks a LOT to all who posted back (what a supportive community!
This was my first post) especially to Kris.
On Friday 10 December 2004 18:21, Kris Deugau wrote:
> Yassen Damyanov wrote:
> >
> > debug: running meta tests; score so far=5.53
> > debug: auto-learn? ham=0.2, spam=8, body-hits=4.166, head-hits=1.364
> ^^^^^^^
> This isn't exactly obvious, but this may be part of your problem. I've
> had trouble in the past with Bayes learning very low-scoring spam as ham
> - so I lowered the autolearn-as-ham threshold to -0.1.
I examined the bayesian manual learning and it does seem perfect.
I came to a conclusion that some real spams got scored very low and
poisoned the Bayes db, so it started to make mistakes and thus
poisoned itself even more, then made worse mistakes, ... etc.
I switched off the auto_learning (not needed IMHO when we have
regular manual learning sessions) and then deleted the old database
and rerun the manual learning script.
Expecting it all to be better now. Thanks again!
BTW, how to interpret things like "tests=BAYES_56" or "tests=BAYES_00"
in the X-Spam-Status header? (Sorry for my ignorance.)
Yassen
--
Yassen Damyanov
phone: +359-32-968-903
email: yd@troyer.co.at
ICQ# : 169382108
web : www.troyer-is.com
Re: Sudden and strange SA Score reduction
Posted by Kris Deugau <kd...@vianet.ca>.
Yassen Damyanov wrote:
>
> Hi SA User List,
>
> Here's my case: postfix + amavisd-new + SpamAssassin 2.64
> working on a Gentoo Linux box, serving as a mail server for
> serveral virtual domains.
>
> Some SpamAssassin details: Bayes learning activated recently,
> based on about 300 spam mails and 200 ham mails, which accumulate
> in IMAP folders and are scanned using sa-learn via cron job three
> times a day.
>
> It all seems to work, but I see SA passing through some obvious spam,
> so I decided to look. And what I dicovered was very surprising for me:
> SA computes the score well, but suddenly lowers it significantly
> exactly before returning an answer to amavisd.
>
> Here an examples:
> .
> .
> debug: running raw-body-text per-line regexp tests; score so far=4.166
> debug: running uri tests; score so far=4.166
> debug: uri tests: Done uriRE
> debug: running full-text regexp tests; score so far=4.166
> debug: all '*From' addrs: fmyldsyx@AMarseille-207-1-4-127.w81-249.abo.wanadoo.fr
> debug: all '*To' addrs: test@jens-weigel.de kht@troyer.co.at sgospodinova@troyer.co.at vtodorov@troyer.co.at yd@troyer.co.at ydamian
> ov@mail2.troyer.co.at
> debug: forged-HELO: from=media-c.local helo=troyer.co.at by=media-c.de
> debug: forged-HELO: mismatch on HELO: 'troyer.co.at' != 'media-c.local'
> debug: forged-HELO: from=wanadoo.fr helo= by=troyer.co.at
> debug: forged-HELO: mismatch on from: 'media-c.local' != 'troyer.co.at'
> debug: running meta tests; score so far=5.53
> debug: auto-learn? ham=0.2, spam=8, body-hits=4.166, head-hits=1.364
^^^^^^^
This isn't exactly obvious, but this may be part of your problem. I've
had trouble in the past with Bayes learning very low-scoring spam as ham
- so I lowered the autolearn-as-ham threshold to -0.1.
> debug: is spam? score=0.629 required=6.8
> tests=BAYES_00,DATE_IN_PAST_12_24,SARE_ADULT2,SARE_OBFUPORNO
^^^^^^^^
As already mentioned by others, this is your ovbious up-front problem.
Bayes is considering the message to be ham, so there's a pretty big
score reduction. Depending on how long your Bayes db has been live, you
may be able to just learn a spam collection correctly and fix the
problem, or you may have to delete it and start again.
> I suspected the bayesian learning to be blamed... but when checking
> the learning sesssions logs, everyhting is correct, spam and ham are
> perfectly sorted and learning is conducted as appropriate. So I am
> stuck.
Manual learning may not be at fault, but *something* is feeding spam in
as ham. How to you feed mail into sa-learn? Do you just periodically
sa-learn a set of inboxes and spam folders? Do you have hand-sorted
mail folders that get periodically learned? Do you just manually learn
reported mistagged mail (of either variety)?
-kgd
--
Get your mouse off of there! You don't know where that email has been!
Re: Sudden and strange SA Score reduction
Posted by Loren Wilton <lw...@earthlink.net>.
debug: is spam? score=0.629 required=6.8
tests=BAYES_00,DATE_IN_PAST_12_24,SARE_ADULT2,SARE_OBFUPORNO
Then the message is tagged "X-Spam-Status: No, hits=0.6".
This is an obvious adult site adv. and SARE rules do a good job. But why the
score
is lowered at the end?
BAYES_00. That says Bayes is absolutely sure this is HAM, not spam, so
lowers the score by -4 points or so.
It would appear that you have your Bayes database trained backwards or some
such. Probably easiest at this point to simply toss the current database,
recheck the data you are planning on feeding in, and then train up a new
database.
Loren
Re: Sudden and strange SA Score reduction
Posted by Duncan Hill <sa...@nacnud.force9.co.uk>.
On Friday 10 December 2004 14:47, Yassen Damyanov might have typed:
> debug: auto-learn? no: inside auto-learn thresholds
> debug: is spam? score=0.629 required=6.8 tests=BAYES_00,
Your Bayes DB for the amavis install reckoned the mail was -4.9 points.
Re: Sudden and strange SA Score reduction
Posted by Matt Kettler <mk...@comcast.net>.
At 04:47 PM 12/10/2004 +0200, Yassen Damyanov wrote:
>debug: Score set 2 chosen.
>debug: auto-learn? no: inside auto-learn thresholds
>debug: is spam? score=0.629 required=6.8
>tests=BAYES_00,DATE_IN_PAST_12_24,SARE_ADULT2,SARE_OBFUPORNO
>
>Then the message is tagged "X-Spam-Status: No, hits=0.6".
>
>This is an obvious adult site adv. and SARE rules do a good job. But why
>the score
>is lowered at the end?
because it hit BAYES_00.
You should revisit your bayes training.
>If I use spamassassin command line tool for the SAME message, I get very good
>result:
>X-Spam-Status: No, hits=6.5 required=6.8 tests=BEST_PORN,DATE_IN_PAST_12_24,
> SARE_ADULT2,SARE_OBFUPORNO autolearn=no version=2.64
the second time, it did not hit BAYES_ at all. Are you sure you ran it as
the same user, thus with the same bayes DB?