You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Don Newcomer <ne...@dickinson.edu> on 2004/05/03 20:46:22 UTC
Bayes problems
I'm having a problem that are a little similar to one reported earlier
today but not quite the same. After seeing a strange-looking false
positive, I realized that there were no Bayes scores that would have offset
the high score from the rulesets. I did a count on my MailScanner logs and
found that 1/3 of the messages labeled as spam had no Bayes scores. Bayes
is working fine otherwise. I'm running SpamAssassin 2.63 with MailScanner
4.29.7 (soon to be 4.30). Any idea what's happening here? Thanks in
advance.
================================================================================
Don Newcomer Dickinson College
Senior Manager, Systems P.O. Box 1773
newcomer@dickinson.edu Carlisle, PA 17013
Phone: (717) 245-1256
FAX: (717) 245-1690
Re: Bayes problems
Posted by Don Newcomer <ne...@dickinson.edu>.
That makes sense. We're still very much in the testing phase of SA so I
don't really have a good way for users to report false positives and
negatives. A few weeks ago I saved all spam to a folder, weeded out any
FPs, and learned it all as spam. For ham I grabbed my mailbox and a few
others. Mail's a pretty touchy subject so I don't know how receptive
people would be to us grabbing their legit mail to train Bayes. I'm doing
expiry runs daily. Should I consider re-learning my spam corpus that I've
saved up?
Don
On Mon, 3 May 2004, Matt Kettler wrote:
> At 02:46 PM 5/3/04 -0400, Don Newcomer wrote:
> >m having a problem that are a little similar to one reported earlier
> >today but not quite the same. After seeing a strange-looking false
> >positive, I realized that there were no Bayes scores that would have offset
> >the high score from the rulesets. I did a count on my MailScanner logs and
> >found that 1/3 of the messages labeled as spam had no Bayes scores. Bayes
> >is working fine otherwise. I'm running SpamAssassin 2.63 with MailScanner
> >4.29.7 (soon to be 4.30). Any idea what's happening here? Thanks in
> >advance.
>
> Lack of bayes score isn't abnormal. Normally it's a sign your bayes
> training is lacking.
>
> If there is nearly and exact 50/50 probability of spam ( between 49.99%
> and 50.01%), or if there are no token hits at all, no bayes score will be
> reported by SA.
>
> However it's a bit unusual to have 1/3 of your messages not hit bayes at
> all, unless your training is VERY deficient. How often do you train
> manually? What do the stats look like if you do a sa-learn --dump magic?
>
>
>
>
Re: Bayes problems
Posted by Matt Kettler <mk...@comcast.net>.
At 02:46 PM 5/3/04 -0400, Don Newcomer wrote:
>m having a problem that are a little similar to one reported earlier
>today but not quite the same. After seeing a strange-looking false
>positive, I realized that there were no Bayes scores that would have offset
>the high score from the rulesets. I did a count on my MailScanner logs and
>found that 1/3 of the messages labeled as spam had no Bayes scores. Bayes
>is working fine otherwise. I'm running SpamAssassin 2.63 with MailScanner
>4.29.7 (soon to be 4.30). Any idea what's happening here? Thanks in
>advance.
Lack of bayes score isn't abnormal. Normally it's a sign your bayes
training is lacking.
If there is nearly and exact 50/50 probability of spam ( between 49.99%
and 50.01%), or if there are no token hits at all, no bayes score will be
reported by SA.
However it's a bit unusual to have 1/3 of your messages not hit bayes at
all, unless your training is VERY deficient. How often do you train
manually? What do the stats look like if you do a sa-learn --dump magic?