You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Charles Sprickman <sp...@bway.net> on 2015/06/29 01:14:40 UTC
Verify Bayes is functioning
This is hopefully easier than I’m thinking it is.
We’ve been running without bayes for a very long time and I thought I’d give
it a shot again with autolearning to see if it’s helpful. The last time I
touched it was 2.6.something and we had spam scanning spread across four
servers and I don’t believe bayes was capable of storing tokens in mysql.
We’re now down to two boxes, and I’ve configured mysql storage.
In a nutshell, this is what I’ve got:
v320.pre:
loadplugin Mail::SpamAssassin::Plugin::Bayes
v310.pre:
loadplugin Mail::SpamAssassin::Plugin::AutoLearnThreshold
local.cf:
# Bayes settings
bayes_store_module Mail::SpamAssassin::BayesStore::MySQL
bayes_sql_dsn DBI:mysql:spamass:10.88.77.x
bayes_sql_username spamass
bayes_sql_password SECRET
use_bayes 1
bayes_auto_learn 1
bayes_auto_learn_threshold_nonspam 0.1
bayes_auto_learn_threshold_spam 12.0
bayes_journal_max_size 102400
bayes_expiry_max_db_size 150000
bayes_auto_expire 1
bayes_learn_to_journal 1
If I look in the mysql db, I see plenty of entries.
If I run sa-learn and ask it to dump some info, that works:
[root@spam-b /usr/local/etc/mail/spamassassin]# sa-learn --username=spork@bway.net --dump magic
0.000 0 3 0 non-token data: bayes db version
0.000 0 7 0 non-token data: nspam
0.000 0 243 0 non-token data: nham
0.000 0 56976 0 non-token data: ntokens
0.000 0 1435355510 0 non-token data: oldest atime
0.000 0 1435529521 0 non-token data: newest atime
0.000 0 0 0 non-token data: last journal sync atime
0.000 0 0 0 non-token data: last expiry atime
0.000 0 0 0 non-token data: last expire atime delta
0.000 0 0 0 non-token data: last expire reduction count
But I never see any bayes rule hits in the headers of my emails. I have in
my personal sql prefs the following:
add_header ham Bayes-Toks _TOKENSUMMARY_
That’s always empty.
So, is bayes working? I clearly am adding tokens and autolearn is happening.
But I’m not convinced any bayes-related rules are happening. What’s the
procedure to actually test this?
Glancing at the sa-learn dump, I’m mighty suspicious of the low number of
autolearn spam messages ( 7 0 non-token data: nspam), but I may be
mis-reading that.
Any pointers to debug further?
I feel like something very obvious is amiss here.
Thanks,
Charles
Re: Verify Bayes is functioning
Posted by RW <rw...@googlemail.com>.
On Sun, 28 Jun 2015 19:14:40 -0400
Charles Sprickman wrote:
> 0.000 0 7 0 non-token data: nspam
> ...
> But I never see any bayes rule hits in the headers of my emails. I
> have in my personal sql prefs the following:
bayes_min_ham_num (Default: 200)
bayes_min_spam_num (Default: 200)
To be accurate, the Bayes system does not activate until a
certain number of ham (non-spam) and spam have been learned.
The default is 200 of each ham and spam, but you can tune these
up or down with these two settings.
Re: Verify Bayes is functioning
Posted by RW <rw...@googlemail.com>.
On Tue, 30 Jun 2015 11:41:23 -0500 (CDT)
David B Funk wrote:
> On Tue, 30 Jun 2015, Charles Sprickman wrote:
>
> > Reindl Harald <h....@thelounge.net> wrote:
> >
> >>
> >> it can't with just 7 spam samples
> >
> > Oh my.
> >
> > So apparently my pre-spamassassin filtering is keeping all the best
> > spam away from spamassassin.
> >
> > My autolearn threshold is set to 12 for spam. I?ll take it down a
> > few notches, feed in some spam from elsewhere, and see what happens.
> >
> > Thanks for the pointer, it simply didn?t occur to me that it?s not
> > going to do anything without a large enough spam/ham sample size.
>
> I believe that the floor for the autolearn-spam threshold is
> internally hard coded to 12 (with a 6/6 split) so it won't do any
> good to drop your config setting below 12.
It's 3 point's from the headers and 3 points from the body.
AFAIK there's no hard-coded 6 point limit, and it isn't strictly
speaking implied by the 3+3 condition.
Re: Verify Bayes is functioning
Posted by David B Funk <db...@engineering.uiowa.edu>.
On Tue, 30 Jun 2015, Charles Sprickman wrote:
> Reindl Harald <h....@thelounge.net> wrote:
>
>>
>> it can't with just 7 spam samples
>
> Oh my.
>
> So apparently my pre-spamassassin filtering is keeping all the best spam
> away from spamassassin.
>
> My autolearn threshold is set to 12 for spam. I’ll take it down a few
> notches, feed in some spam from elsewhere, and see what happens.
>
> Thanks for the pointer, it simply didn’t occur to me that it’s not going to
> do anything without a large enough spam/ham sample size.
I believe that the floor for the autolearn-spam threshold is internally hard
coded to 12 (with a 6/6 split) so it won't do any good to drop your config setting
below 12. Even the tflags "autolearn_force" action is limited by this coding,
so if the spam doesn't score at least 12 it will never get autolearned.
Either set up some kind of external path to feed known spam directly to
"sa-learn --spam" or add some special rules to take known spam and slam
them with a grob'o points.
I ran into this issue when trying to force-auto-learn spam-trap fodder.
--
Dave Funk University of Iowa
<dbfunk (at) engineering.uiowa.edu> College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{
Re: Verify Bayes is functioning
Posted by Bob Proulx <bo...@proulx.com>.
Charles Sprickman wrote:
> So apparently my pre-spamassassin filtering is keeping all the best spam
> away from spamassassin.
If you have pre-spamassassin filtering that is already classifying
message as spam then consider feeing those spam messages to
spamassassin's "sa-learn --spam" for Bayes training.
Bob
Re: Verify Bayes is functioning
Posted by Charles Sprickman <sp...@bway.net>.
Reindl Harald <h....@thelounge.net> wrote:
>
> Am 29.06.2015 um 01:14 schrieb Charles Sprickman:
>> If I run sa-learn and ask it to dump some info, that works:
>>
>> [root@spam-b /usr/local/etc/mail/spamassassin]# sa-learn --username=spork@bway.net --dump magic
>> 0.000 0 3 0 non-token data: bayes db version
>> 0.000 0 7 0 non-token data: nspam
>> 0.000 0 243 0 non-token data: nham
>> 0.000 0 56976 0 non-token data: ntokens
>> 0.000 0 1435355510 0 non-token data: oldest atime
>> 0.000 0 1435529521 0 non-token data: newest atime
>> 0.000 0 0 0 non-token data: last journal sync atime
>> 0.000 0 0 0 non-token data: last expiry atime
>> 0.000 0 0 0 non-token data: last expire atime delta
>> 0.000 0 0 0 non-token data: last expire reduction count
>>
>> But I never see any bayes rule hits in the headers of my emails
>
> it can't with just 7 spam samples
Oh my.
So apparently my pre-spamassassin filtering is keeping all the best spam
away from spamassassin.
My autolearn threshold is set to 12 for spam. I’ll take it down a few
notches, feed in some spam from elsewhere, and see what happens.
Thanks for the pointer, it simply didn’t occur to me that it’s not going to
do anything without a large enough spam/ham sample size.
Charles
Re: Verify Bayes is functioning
Posted by Reindl Harald <h....@thelounge.net>.
Am 29.06.2015 um 01:14 schrieb Charles Sprickman:
> If I run sa-learn and ask it to dump some info, that works:
>
> [root@spam-b /usr/local/etc/mail/spamassassin]# sa-learn --username=spork@bway.net --dump magic
> 0.000 0 3 0 non-token data: bayes db version
> 0.000 0 7 0 non-token data: nspam
> 0.000 0 243 0 non-token data: nham
> 0.000 0 56976 0 non-token data: ntokens
> 0.000 0 1435355510 0 non-token data: oldest atime
> 0.000 0 1435529521 0 non-token data: newest atime
> 0.000 0 0 0 non-token data: last journal sync atime
> 0.000 0 0 0 non-token data: last expiry atime
> 0.000 0 0 0 non-token data: last expire atime delta
> 0.000 0 0 0 non-token data: last expire reduction count
>
> But I never see any bayes rule hits in the headers of my emails
it can't with just 7 spam samples