You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Andrew Talbot <an...@gmail.com> on 2013/05/01 20:01:39 UTC
Bayes Autolearning
Hey All -
I'm about to set up Bayes on one of our mail servers. A lot of the
documentation says that I need to manually sift through a few hundred
messages and classify them to 'teach' the filter, and it sounds like I may
need to do that on an ongoing basis.
That is not a very plausible solution - our servers process about 2million
messages a day.
Does Bayes start out with a completely blank slate? That is, if I never have
it learn anything from my servers, will it still be pulling from something
already defined?
Can I set it to autolearn and leave it be? Or will it require continual
maintenance and manual message feeding?
Any suggestions any of you have for a Bayes newbie - about what I just asked
or otherwise - would be very much appreciated J
Re: Bayes Autolearning
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Wed, 2013-05-01 at 14:35 -0400, Andrew Talbot wrote:
> Thank you for that!
>
> Off-list you mentioned that you don't need to set the cron/expire because of
> Redis features; why is it commented out here?
Because it is a comment about a cron job, followed by an actual
'sa-learn' command. Commented out, consider it documentation.
Unlike everything above, which are M::SA::Conf configuration options
suitable for a .cf file. With the occasional comment for documentation.
> > # bayes.cf
> >
> > use_bayes 1
> > bayes_auto_learn 1
> > bayes_auto_expire 0
> >
> > bayes_learn_to_journal 0
> >
> > # Dont' want to wait for the deault 200 hams/spams bayes_min_ham_num
> > 20 bayes_min_spam_num 20
> > # cron weekly
> > # sa-learn --force-expire
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
RE: Bayes Autolearning
Posted by Andrew Talbot <an...@gmail.com>.
Thank you for that!
Off-list you mentioned that you don't need to set the cron/expire because of
Redis features; why is it commented out here?
> -----Original Message-----
> From: Axb [mailto:axb.lists@gmail.com]
> Sent: Wednesday, May 01, 2013 2:14 PM
> To: users@spamassassin.apache.org
> Subject: Re: Bayes Autolearning
>
> On 05/01/2013 08:01 PM, Andrew Talbot wrote:
>
> > Any suggestions any of you have for a Bayes newbie - about what I just
> > asked or otherwise - would be very much appreciated.
>
> I advocate autolearning as it has always worked fine for me.
> Can take a bit longer to see good results but with some tuning I can sit
back
> and hear it purr and not worry about collecting ham and spam and training,
> which under certain circumstances may even be impossible.
>
> Before moving on to Redis, these were my bayes settings
>
> # bayes.cf
>
> use_bayes 1
> bayes_auto_learn 1
> bayes_auto_expire 0
>
> bayes_learn_to_journal 0
>
> # Dont' want to wait for the deault 200 hams/spams bayes_min_ham_num
> 20 bayes_min_spam_num 20
>
> bayes_auto_learn_threshold_nonspam -1.0
> bayes_auto_learn_threshold_spam 15.0
>
>
> # FILE BASED
> # mkdir /etc/bayes
> bayes_path /etc/mail/spamassassin/bayes/bayes
>
> # Check permsisions/modify if needed
> #bayes_file_mode 0666
>
> bayes_expiry_max_db_size 350000
> # SDBM is faster than other r/w DBs
> bayes_store_module Mail::SpamAssassin::BayesStore::SDBM
>
> # cron weekly
> # sa-learn --force-expire
RE: Bayes Autolearning
Posted by Andrew Talbot <an...@gmail.com>.
Hey there, thanks for responding. That's an interesting point.
Are you saying I should not use autolearning at all?
I don't have any way to review a large corpus of messages because we don't
have access to them - after they run through our servers they are sent on,
and the text of the message is not stored on our server.
Man, I wish there was an easier way to feed Bayes an initial set of spam/ham
to teach it properly .. I've been told that letting it autolearn for a few
hours/days would make it learn well enough though.
If only our mail server only got 100 messages a day - then I could just
manually mark them! :)
> -----Original Message-----
> From: RW [mailto:rwmaillists@googlemail.com]
> Sent: Wednesday, May 01, 2013 6:24 PM
> To: users@spamassassin.apache.org
> Subject: Re: Bayes Autolearning
>
> On Wed, 01 May 2013 22:02:43 +0100
> Steve Freegard wrote:
>
> > On 01/05/13 19:40, Andrew Talbot wrote:
> > > Hi, Seve -
> > >
> > > Thanks for your response. Is that just for performance reasons?
> > >
> >
> > Performance is one of the things that bayes_auto_learn_on_error 1 will
> > give you. It means that if the message was already considered spam by
> > Bayes, then the message won't be autolearnt again which means
> > a bit less IO. It will also result in the Bayes databases being
> > smaller as it is likely that with this option that less tokens will be
> > present overall which will also save disk IO and space.
> >
> > But the key reason I like this option is that it doesn't allow bayes
> > to overtrain in one direction (e.g. spam or ham). It only autolearns
> > when Bayes either has the wrong result or isn't sure which IMO has to
> > be better for accuracy in the long run.
>
> The evidence from trials with Bogofilter (which is similar to Bayes)
showed
> that initially train-on-everything significantly outperforms
train-on-error. The
> latter asymptotically catches up after thousands of errors. It seems that
the
> most important thing is to learn a few thousand hams and spams by any
> means; and train-on-error can take a long time to get there. For this
reason
> DSPAM only allows train-on-error when 2500 hams have been learned.
>
> There *may* be advantages to train-on-error after this in preventing BAYES
> becoming insensitive to learning.
>
> The chief problem with autolearning is learning ham. If you set a positive
> threshold you end-up learning a lot of spam as ham, if you set a negative
> threshold you effectively turn-over ham training to the DNS whitelists
since
> they are the only tests with significant negative scores that aren't
excluded
> from autolearning. Any problems with miss-learning are likely to be
> exacerbated by train-on-error.
>
> If I had to use autolearning I'd mark the DNS whitelists as noautolearn
and
> write some negative-scoring, site-specific rules.
Re: Bayes Autolearning
Posted by RW <rw...@googlemail.com>.
On Wed, 01 May 2013 22:02:43 +0100
Steve Freegard wrote:
> On 01/05/13 19:40, Andrew Talbot wrote:
> > Hi, Seve -
> >
> > Thanks for your response. Is that just for performance reasons?
> >
>
> Performance is one of the things that bayes_auto_learn_on_error 1
> will give you. It means that if the message was already considered
> spam by Bayes, then the message won't be autolearnt again which means
> a bit less IO. It will also result in the Bayes databases being
> smaller as it is likely that with this option that less tokens will
> be present overall which will also save disk IO and space.
>
> But the key reason I like this option is that it doesn't allow bayes
> to overtrain in one direction (e.g. spam or ham). It only autolearns
> when Bayes either has the wrong result or isn't sure which IMO has to
> be better for accuracy in the long run.
The evidence from trials with Bogofilter (which is similar to Bayes)
showed that initially train-on-everything significantly outperforms
train-on-error. The latter asymptotically catches up after thousands
of errors. It seems that the most important thing is to learn a few
thousand hams and spams by any means; and train-on-error can take a
long time to get there. For this reason DSPAM only allows
train-on-error when 2500 hams have been learned.
There *may* be advantages to train-on-error after this in preventing
BAYES becoming insensitive to learning.
The chief problem with autolearning is learning ham. If you set a
positive threshold you end-up learning a lot of spam as ham, if you set
a negative threshold you effectively turn-over ham training to the DNS
whitelists since they are the only tests with significant negative
scores that aren't excluded from autolearning. Any problems with
miss-learning are likely to be exacerbated by train-on-error.
If I had to use autolearning I'd mark the DNS whitelists as noautolearn
and write some negative-scoring, site-specific rules.
Re: Bayes Autolearning
Posted by Steve Freegard <st...@fsl.com>.
On 01/05/13 19:40, Andrew Talbot wrote:
> Hi, Seve -
>
> Thanks for your response. Is that just for performance reasons?
>
Performance is one of the things that bayes_auto_learn_on_error 1 will
give you. It means that if the message was already considered spam by
Bayes, then the message won't be autolearnt again which means a bit less
IO. It will also result in the Bayes databases being smaller as it is
likely that with this option that less tokens will be present overall
which will also save disk IO and space.
But the key reason I like this option is that it doesn't allow bayes to
overtrain in one direction (e.g. spam or ham). It only autolearns when
Bayes either has the wrong result or isn't sure which IMO has to be
better for accuracy in the long run.
Kind regards,
Steve.
RE: Bayes Autolearning
Posted by Andrew Talbot <an...@gmail.com>.
Hi, Seve -
Thanks for your response. Is that just for performance reasons?
> -----Original Message-----
> From: Steve Freegard [mailto:steve.freegard@fsl.com]
> Sent: Wednesday, May 01, 2013 2:24 PM
> To: users@spamassassin.apache.org
> Subject: Re: Bayes Autolearning
>
> All good advice there from Axb; the only thing I'd add to that is:
>
> bayes_auto_learn_on_error 1
>
> Which prevents Bayes from over-training when the classifier already agrees
> with what the autolearn is trying to train on.
>
> Cheers,
> Steve.
>
> On 01/05/13 19:14, Axb wrote:
> > On 05/01/2013 08:01 PM, Andrew Talbot wrote:
> >
> >> Any suggestions any of you have for a Bayes newbie - about what I
> >> just asked or otherwise - would be very much appreciated.
> >
> > I advocate autolearning as it has always worked fine for me.
> > Can take a bit longer to see good results but with some tuning I can
> > sit back and hear it purr and not worry about collecting ham and spam
> > and training, which under certain circumstances may even be impossible.
> >
> > Before moving on to Redis, these were my bayes settings
> >
> > # bayes.cf
> >
> > use_bayes 1
> > bayes_auto_learn 1
> > bayes_auto_expire 0
> >
> > bayes_learn_to_journal 0
> >
> > # Dont' want to wait for the deault 200 hams/spams bayes_min_ham_num
> > 20 bayes_min_spam_num 20
> >
> > bayes_auto_learn_threshold_nonspam -1.0
> > bayes_auto_learn_threshold_spam 15.0
> >
> >
> > # FILE BASED
> > # mkdir /etc/bayes
> > bayes_path /etc/mail/spamassassin/bayes/bayes
> >
> > # Check permsisions/modify if needed
> > #bayes_file_mode 0666
> >
> > bayes_expiry_max_db_size 350000
> > # SDBM is faster than other r/w DBs
> > bayes_store_module Mail::SpamAssassin::BayesStore::SDBM
> >
> > # cron weekly
> > # sa-learn --force-expire
> >
> >
>
Re: Bayes Autolearning
Posted by Steve Freegard <st...@fsl.com>.
All good advice there from Axb; the only thing I'd add to that is:
bayes_auto_learn_on_error 1
Which prevents Bayes from over-training when the classifier already
agrees with what the autolearn is trying to train on.
Cheers,
Steve.
On 01/05/13 19:14, Axb wrote:
> On 05/01/2013 08:01 PM, Andrew Talbot wrote:
>
>> Any suggestions any of you have for a Bayes newbie - about what I just
>> asked
>> or otherwise - would be very much appreciated.
>
> I advocate autolearning as it has always worked fine for me.
> Can take a bit longer to see good results but with some tuning I can
> sit back and hear it purr and not worry about collecting ham and spam
> and training, which under certain circumstances may even be impossible.
>
> Before moving on to Redis, these were my bayes settings
>
> # bayes.cf
>
> use_bayes 1
> bayes_auto_learn 1
> bayes_auto_expire 0
>
> bayes_learn_to_journal 0
>
> # Dont' want to wait for the deault 200 hams/spams
> bayes_min_ham_num 20
> bayes_min_spam_num 20
>
> bayes_auto_learn_threshold_nonspam -1.0
> bayes_auto_learn_threshold_spam 15.0
>
>
> # FILE BASED
> # mkdir /etc/bayes
> bayes_path /etc/mail/spamassassin/bayes/bayes
>
> # Check permsisions/modify if needed
> #bayes_file_mode 0666
>
> bayes_expiry_max_db_size 350000
> # SDBM is faster than other r/w DBs
> bayes_store_module Mail::SpamAssassin::BayesStore::SDBM
>
> # cron weekly
> # sa-learn --force-expire
>
>
Re: Bayes Autolearning
Posted by Axb <ax...@gmail.com>.
On 05/01/2013 08:01 PM, Andrew Talbot wrote:
> Any suggestions any of you have for a Bayes newbie - about what I just asked
> or otherwise - would be very much appreciated.
I advocate autolearning as it has always worked fine for me.
Can take a bit longer to see good results but with some tuning I can
sit back and hear it purr and not worry about collecting ham and spam
and training, which under certain circumstances may even be impossible.
Before moving on to Redis, these were my bayes settings
# bayes.cf
use_bayes 1
bayes_auto_learn 1
bayes_auto_expire 0
bayes_learn_to_journal 0
# Dont' want to wait for the deault 200 hams/spams
bayes_min_ham_num 20
bayes_min_spam_num 20
bayes_auto_learn_threshold_nonspam -1.0
bayes_auto_learn_threshold_spam 15.0
# FILE BASED
# mkdir /etc/bayes
bayes_path /etc/mail/spamassassin/bayes/bayes
# Check permsisions/modify if needed
#bayes_file_mode 0666
bayes_expiry_max_db_size 350000
# SDBM is faster than other r/w DBs
bayes_store_module Mail::SpamAssassin::BayesStore::SDBM
# cron weekly
# sa-learn --force-expire