How to use sa-learn to manuall train?

[SamW <sw...@svtinc.com>]

Here is how I use the sa-learn to train the bayes filter in version SA 2.63

I asked our users to send a mail to spam@mydomain.com if the users see a 
spam message. If they see a non spam (ham) message they send a mail to 
notspam@mydomain.com. Note that the messages that the user receives is 
already tagged with X-Spam-* headers

I then run following 2 commands, to train the sa 4 times a day through a 
crontab.

	/usr/local/bin/sa-learn --spam --mbox /var/mail/spam
	/usr/local/bin/sa-learn --spam --mbox /var/mail/notspam

Then I also feed the same messages to razor
	/usr/local/bin/razor-report /var/mail/spm

sa-learning script seems to run, and learn. But here are my concerns,

Every mail is tagged, with x-spam-* headers that I feed to spam filter
Every mail which SPAM is also in most cases has its Subject Field changed. 
with ***SPAM**

My Questions

1. Should I remove spam headers and remove the ***SPAM*** from Subject 
field before I feed mail to sa-learn?
2. Should I remove Spam headers before I feed them to razor ?

IS THERE ANY BETTER WAY OR HOW-TO SOME WHERE I CAN START TO USE? How do you 
folks use sa-learn?

Thank you in advance.

- SamW

Re: How to use sa-learn to manuall train?

[Steven Dickenson <st...@mrchuckles.net>]

SamW wrote:
> I asked our users to send a mail to spam@mydomain.com if the users see a 
> spam message. If they see a non spam (ham) message they send a mail to 
> notspam@mydomain.com. Note that the messages that the user receives is 
> already tagged with X-Spam-* headers

This could be a bad thing depending on how you have them send the 
messages to these addresses.  Forwarding them will not do it, sa-learn 
needs the original headers intact.  For MS Outlook, you need to open the 
message, go the action menu, and choose resend.

> 1. Should I remove spam headers and remove the ***SPAM*** from Subject 
> field before I feed mail to sa-learn?

SA-Learn will know how to deal with already tagged messages, so this is 
okay.

> 2. Should I remove Spam headers before I feed them to razor ?
> 
> IS THERE ANY BETTER WAY OR HOW-TO SOME WHERE I CAN START TO USE? How do 
> you folks use sa-learn?

If you're reporting to online authorities, look into using the --report 
option of SA.  This has the benefit of learning into the Bayes database 
at the same time.

I only manually learn false positives.  I let autolearning handle the 
rest, and it usually does a pretty good job.

Steven
-- 
Steven Dickenson <st...@mrchuckles.net>
http://www.mrchuckles.net

Re: How to use sa-learn to manuall train?

[Pat <cy...@yahoo.com>]

I have several comments about the way you are using
spamassassin.
First of all, the method you are using is only good if
you are using a central bayes database for known spam.
Otherwise you need to run sa-learn for each user so
that -their- database stays current.
Second, you do need all original headers present on
the text you feed to sa-learn. Rewritten emails (like
forwards that you seem to use) will not hold any of
those headers.
Third, reporting messages to dcc/razor/pyzor etc
doesn't require you to remove any of the spamassassin
markups. The utilities do so automatically before
sending the messages to the servers.

For your purposes I would recommend adding rules to
procmail to properly sort emails (assuming this is
what you are using). I will give you an example
similar to what i am implementing.
if the header contains X-Spam-Status: Yes it goes in a
spambox, if it has X-Spam-Status: No it goes in the
ham box. 

then you might need to go through these boxes to make
sure you didn't mark ham as spam and spam as ham etc.

My actual trick is a little more complicated. I have
multiple boxes per user: ham, not_ham, spam, not_spam.
then i go through them so that ham only contains ham,
not_ham only contains incorrectly passed spam, spam
contains only spam, and not_spam is only incorrectly
flagged ham. then once everything is accurate i learn
all the files for the given user.

I hope this helps you develop your system.
Pat
--- SamW <sw...@svtinc.com> wrote:
> Here is how I use the sa-learn to train the bayes
> filter in version SA 2.63
> 
> I asked our users to send a mail to
> spam@mydomain.com if the users see a 
> spam message. If they see a non spam (ham) message
> they send a mail to 
> notspam@mydomain.com. Note that the messages that
> the user receives is 
> already tagged with X-Spam-* headers
> 
> I then run following 2 commands, to train the sa 4
> times a day through a 
> crontab.
> 
> 	/usr/local/bin/sa-learn --spam --mbox
> /var/mail/spam
> 	/usr/local/bin/sa-learn --spam --mbox
> /var/mail/notspam
> 
> Then I also feed the same messages to razor
> 	/usr/local/bin/razor-report /var/mail/spm
> 
> sa-learning script seems to run, and learn. But here
> are my concerns,
> 
> Every mail is tagged, with x-spam-* headers that I
> feed to spam filter
> Every mail which SPAM is also in most cases has its
> Subject Field changed. 
> with ***SPAM**
> 
> My Questions
> 
> 1. Should I remove spam headers and remove the
> ***SPAM*** from Subject 
> field before I feed mail to sa-learn?
> 2. Should I remove Spam headers before I feed them
> to razor ?
> 
> IS THERE ANY BETTER WAY OR HOW-TO SOME WHERE I CAN
> START TO USE? How do you 
> folks use sa-learn?
> 
> Thank you in advance.
> 
> - SamW
> 


__________________________________
Do you Yahoo!?
Yahoo! Tax Center - File online by April 15th
http://taxes.yahoo.com/filing.html