You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Micah Anderson <mi...@riseup.net> on 2009/05/02 17:28:28 UTC
bayes training doesn't seem to have any affect
I got a phish message that was understood by bayes as:
-2.6 BAYES_00 BODY: Bayesian spam probability is 0 to 1%
[score: 0.0000]
So I traiend with spamc -L spam but even after that I am still getting
BAYES_00. Shouldn't the training have bumped that score up?
Thanks for any info,
micah
Re: bayes training doesn't seem to have any affect
Posted by Micah Anderson <mi...@riseup.net>.
Karsten Bräckelmann <gu...@rudersport.de> writes:
>> This shows me that I have no idea what these magic things are :) Does
>> this tell you anything useful?
>
>> 0.000 0 6798614 0 non-token data: nspam
>> 0.000 0 19136753 0 non-token data: nham
>
> That's quite a lot of ham compared to the spam... Does that really
> reflect your mail instream?
I would suspect not, since we probably get more spam than
non-spam. However, perhaps the spamassassin autolearning caused this?
Perhaps the DB is so out of whack, I should just reset it from scratch
and try it again. Its a lot of data to loose and I am not sure exactly
the right way to do that... so I'd be somewhat reluctant to do so. Might
be better if I could clean it out some.
> 19 M hams learned and an SQL Bayes storage backend. Site wide. Do you
> trust your users? Any chance some of them are training badly? At worst
No, I don't trust my users. In fact because of that we moved from doing
site-wide training to selected users who can demonstrate that they
understand how to train. Perhaps these numbers are legacy from before we
switched to this method.
thanks,
micah
Re: bayes training doesn't seem to have any affect
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Mon, 2009-05-04 at 02:02 -0400, Micah Anderson wrote:
> Dave Walker <Da...@ubuntu.com> writes:
> > Micah Anderson wrote:
> > > I got a phish message that was understood by bayes as:
> > >
> > > -2.6 BAYES_00 BODY: Bayesian spam probability is 0 to 1%
> > > [score: 0.0000]
> > >
> > > So I traiend with spamc -L spam but even after that I am still getting
> > > BAYES_00. Shouldn't the training have bumped that score up?
Yeah, generally speaking. However, every now and then, I too see some
particular spam that resists any training. That is, later spam still
scores (regarding Bayes) way too neutral for my taste. ;)
> > To see what is really going on run "$ spamassassin -D <
> > /path/to/the/email > /dev/null", and see if you can learn anything as to
> > why it's not working as expected.
>
> Indeed, when I do this, I find these bayes related log entries:
>
> [13244] dbg: bayes: corpus size: nspam = 6798614, nham = 19136735
> [13244] dbg: bayes: tok_get_all: token count: 175
> [13244] dbg: bayes: score = 0
Use -D bayes rather than plain -D and check the entire output. I just
hope that your (Pg?)SQL BayesStore backend dumps the tokens just like
the DBM one does.
Since it is phish -- specifically targeted at your site? Any chance they
managed to use a lot of internal, really hammy looking tokens? Possibly
even originating from inside your network?
> This shows me that I have no idea what these magic things are :) Does
> this tell you anything useful?
> 0.000 0 6798614 0 non-token data: nspam
> 0.000 0 19136753 0 non-token data: nham
That's quite a lot of ham compared to the spam... Does that really
reflect your mail instream?
19 M hams learned and an SQL Bayes storage backend. Site wide. Do you
trust your users? Any chance some of them are training badly? At worst
even "rescuing" phish from the spam folder, falling for it, responding
with their credentials and *learning* the phish, believing it was
classified incorrectly?
The latter sure will explain why you got a hard time re-training *one*
sample as spam -- if you got it learned >1 times as ham...
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: bayes training doesn't seem to have any affect
Posted by Theo Van Dinter <fe...@apache.org>.
On Tue, May 5, 2009 at 5:40 PM, Micah Anderson <mi...@riseup.net> wrote:
>> Eh? Last journal sync atime is Jan 1 1970?
>> Try running: sa-learn --sync
>
> Doesn't seem to change the 'last journal sync atime' from 0.
[...]
> I'm using a mysql DB and I've got the following set in my local.cf:
SQL Bayes DBs don't have journals, so no last sync time is expected. fyi.
Re: bayes training doesn't seem to have any affect
Posted by Micah Anderson <mi...@riseup.net>.
Adam Katz <an...@khopis.com> writes:
> Micah Anderson wrote:
>>> Also, to see how experienced your Bayes knowledge is - use "$ sa-leanrn
>>> --dump magic"
>>
>> This shows me that I have no idea what these magic things are :) Does
>> this tell you anything useful?
>>
>> 0.000 0 3 0 non-token data: bayes db version
>> 0.000 0 6798614 0 non-token data: nspam
>> 0.000 0 19136753 0 non-token data: nham
>> 0.000 0 1063157695 0 non-token data: ntokens
>> 0.000 0 1241301616 0 non-token data: oldest atime
>> 0.000 0 1241416889 0 non-token data: newest atime
>> 0.000 0 0 0 non-token data: last journal sync atime
>> 0.000 0 1241344830 0 non-token data: last expiry atime
>> 0.000 0 43200 0 non-token data: last expire atime delta
>> 0.000 0 496607 0 non-token data: last expire reduction count
>
> Eh? Last journal sync atime is Jan 1 1970?
> Try running: sa-learn --sync
Doesn't seem to change the 'last journal sync atime' from 0.
> If that helps, put it in your nightly SpamAssassin cron job
> (and/or revisit your custom teaching scripts).
In fact, I've been running that from cron every night.
I'm using a mysql DB and I've got the following set in my local.cf:
# We want to expire via cronjob, rather than having one of our spamd
# children do it.
bayes_auto_expire 0
# no affect
bayes_learn_to_journal 0
> A quick primer (since this doesn't really exist anywhere...): The
> three zeroed columns are always zero.
>
> bayes db version is self-explanatory.
> nspam is the number of spam messages on record. bayes needs >200.
Should be fine: 6798649
> nham is the number of ham messages on record. bayes needs >200.
Also should be fine: 19160960
> ntokens is the number of 'words' noted in the system.
lots of tokens: 1065483803
> oldest atime is the oldest access time of the oldest token (I think).
I've got 1241474416 which would be Mon May 4 15:00:16 PDT 2009
which is just yesterday... that doesn't seem right that this would be
the oldest access time, especially for 1065483803 tokens!
> the rest of the times should be self-explanatory.
> last expire reduction count is the number of tokens removed from the
> last expiration run (I think).
Ok, that seems to be counting, so something is being expired:
0.000 0 840628 0 non-token data: last expire reduction count
This is all very interesting info, I appreciate the
explanation. However, my original question still stands.
micah
Re: bayes training doesn't seem to have any affect
Posted by Adam Katz <an...@khopis.com>.
Micah Anderson wrote:
>> Also, to see how experienced your Bayes knowledge is - use "$ sa-leanrn
>> --dump magic"
>
> This shows me that I have no idea what these magic things are :) Does
> this tell you anything useful?
>
> 0.000 0 3 0 non-token data: bayes db version
> 0.000 0 6798614 0 non-token data: nspam
> 0.000 0 19136753 0 non-token data: nham
> 0.000 0 1063157695 0 non-token data: ntokens
> 0.000 0 1241301616 0 non-token data: oldest atime
> 0.000 0 1241416889 0 non-token data: newest atime
> 0.000 0 0 0 non-token data: last journal sync atime
> 0.000 0 1241344830 0 non-token data: last expiry atime
> 0.000 0 43200 0 non-token data: last expire atime delta
> 0.000 0 496607 0 non-token data: last expire reduction count
Eh? Last journal sync atime is Jan 1 1970?
Try running: sa-learn --sync
If that helps, put it in your nightly SpamAssassin cron job
(and/or revisit your custom teaching scripts).
A quick primer (since this doesn't really exist anywhere...): The
three zeroed columns are always zero.
bayes db version is self-explanatory.
nspam is the number of spam messages on record. bayes needs >200.
nham is the number of ham messages on record. bayes needs >200.
ntokens is the number of 'words' noted in the system.
oldest atime is the oldest access time of the oldest token (I think).
the rest of the times should be self-explanatory.
last expire reduction count is the number of tokens removed from the
last expiration run (I think).
All times are in unix time notation (seconds since Jan 1 1970), which
you can decipher with the command: date -d @[TIME]
Re: bayes training doesn't seem to have any affect
Posted by Micah Anderson <mi...@riseup.net>.
Dave Walker <Da...@ubuntu.com> writes:
> Micah Anderson wrote:
>> I got a phish message that was understood by bayes as:
>>
>> -2.6 BAYES_00 BODY: Bayesian spam probability is 0 to 1%
>> [score: 0.0000]
>>
>> So I traiend with spamc -L spam but even after that I am still getting
>> BAYES_00. Shouldn't the training have bumped that score up?
>>
>> Thanks for any info,
>>
> In order for Bayes to actually make a difference, it needs plenty of
> training. It's disabled by default in most installs - unless you have
> at least 200 of both spam and ham taught. This needs to be done
> manually, unless you have autolearn enabled.
Yeah, I've been running this bayes db for a couple years now, so I am
sure I've passed the 200 mark :)
I'm wondering if my bayes DB is too poisoned now and maybe needs to be
reset?
> To see what is really going on run "$ spamassassin -D <
> /path/to/the/email > /dev/null", and see if you can learn anything as to
> why it's not working as expected.
Indeed, when I do this, I find these bayes related log entries:
[13244] dbg: bayes: corpus size: nspam = 6798614, nham = 19136735
[13244] dbg: bayes: tok_get_all: token count: 175
[13244] dbg: bayes: score = 0
> Also, to see how experienced your Bayes knowledge is - use "$ sa-leanrn
> --dump magic"
This shows me that I have no idea what these magic things are :) Does
this tell you anything useful?
0.000 0 3 0 non-token data: bayes db version
0.000 0 6798614 0 non-token data: nspam
0.000 0 19136753 0 non-token data: nham
0.000 0 1063157695 0 non-token data: ntokens
0.000 0 1241301616 0 non-token data: oldest atime
0.000 0 1241416889 0 non-token data: newest atime
0.000 0 0 0 non-token data: last journal sync atime
0.000 0 1241344830 0 non-token data: last expiry atime
0.000 0 43200 0 non-token data: last expire atime delta
0.000 0 496607 0 non-token data: last expire reduction count
micah
Re: bayes training doesn't seem to have any affect
Posted by Dave Walker <Da...@ubuntu.com>.
Micah Anderson wrote:
> I got a phish message that was understood by bayes as:
>
> -2.6 BAYES_00 BODY: Bayesian spam probability is 0 to 1%
> [score: 0.0000]
>
> So I traiend with spamc -L spam but even after that I am still getting
> BAYES_00. Shouldn't the training have bumped that score up?
>
> Thanks for any info,
> micah
>
Hi Micah,
In order for Bayes to actually make a difference, it needs plenty of
training. It's disabled by default in most installs - unless you have
at least 200 of both spam and ham taught. This needs to be done
manually, unless you have autolearn enabled.
To see what is really going on run "$ spamassassin -D <
/path/to/the/email > /dev/null", and see if you can learn anything as to
why it's not working as expected.
Also, to see how experienced your Bayes knowledge is - use "$ sa-leanrn
--dump magic"
HTH.
Kind Regards,
Dave Walker