You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Oenus Tech Services <oe...@oenus.com> on 2007/01/24 17:44:22 UTC
sa-learn on dedicated spamabuse email account
Hi there!
Most of our email is delivered through pop3, so right now bayes
filtering is off. Nevertheless Spamassassin is doing a good job
filtering email, but we want to setup a way for our customers to report
to us undetected spam by forwarding that spam to an
spamabuse@example.com account in our server. If we then point sa-learn
to that inbox, will it work? My concern is that email arriving to that
account is not from the spammer anymore, but from a forwarded mail by
our customer.
TIA,
Ignacio
Re: sa-learn on dedicated spamabuse email account
Posted by Pete Russell <pe...@enitech.com.au>.
You have described why it wont work as good. Using your method the
headers become useless - bayes ill learn only the body/subject content.
You would need to tell bayes to ignore the headers.
With pop3 you only have the option you describe - this may still work ok?
Pete
Oenus Tech Services wrote:
> Thanks, Peter
>
> Yes, but this would not work with our more than 1500 customers that have
> only pop3 access and do not have access to any shared or private folder
> in the servers. We needed to implement some way for these pop3-only
> customers to report spam back to us, and for now we've only thought of
> forwarding spam to an spamabuse account where some scripts could check
> its inbox and do bayesian learning. However, does sa-learn take into
> account that those emails are being forwarded and they're not the
> original source of the spam? I guess not. If that is the case, does
> anybody has come up with a similar idea?
>
> Ignacio
>
>
>
> Peter Russell escribió:
>> See attach python script written by one of the folks on the MailScanner
>> list.
>>
>> Its designed for use with exchange, so i will describe the Exchange
>> usage and you can modify as you see fit to work on your pop3 server.
>>
>> In Exchange 2003 create a public folder called SPAM, give everyone
>> contributor access, not read or edit. Then any user can simply drag spam
>> to the public folder, but no user can see in the folder.
>>
>> Modify the script to suit your environment (Exchange server name and
>> credentials). Make it executable.
>>
>> Now run it. It will scan your public folder called SPAM and learn the
>> contents into bayes, then delete the messages it has learned.
>>
>> The script doesnt seem to run recursively all that well, it maye stop
>> randomly and need to be re run again - if any python scripters see this
>> would you mind having a go fixing this and re posting to the list?
>>
>> Many thanks
>> Pete
>>
>> Oenus Tech Services wrote:
>>> Hi there!
>>>
>>> Most of our email is delivered through pop3, so right now bayes
>>> filtering is off. Nevertheless Spamassassin is doing a good job
>>> filtering email, but we want to setup a way for our customers to report
>>> to us undetected spam by forwarding that spam to an
>>> spamabuse@example.com account in our server. If we then point sa-learn
>>> to that inbox, will it work? My concern is that email arriving to that
>>> account is not from the spammer anymore, but from a forwarded mail by
>>> our customer.
>>>
>>> TIA,
>>>
>>> Ignacio
>>>
>> ------------------------------------------------------------------------
>>
>> #!/usr/bin/env python
>> import commands, os, time
>> import imaplib
>> import sys, re
>> import string, random
>> import StringIO, rfc822
>>
>> # Set required variables
>> PREFS = "/etc/MailScanner/spam.assassin.prefs.conf"
>> TMPFILE = "/var/tmp/salearn.tmp"
>> SALEARN = "/usr/bin/sa-learn"
>> SERVER = "x.x.x.x"
>> USER = "someuserwithaccesstopublicfolder"
>> PASSWORD = "somepassword"
>> LOGFILE = "/var/log/learn.spam.log"
>> log = file(LOGFILE, 'a+')
>> log.write("\n\nTraining SpamAssassin on %s at %s\n" % (time.strftime("%Y-%m-%d"), time.strftime("%H:%M:%S")))
>>
>> # connect to server
>> server = imaplib.IMAP4(SERVER)
>>
>> # login
>> server.login(USER, PASSWORD)
>> server.select("Public Folders/Spam")
>>
>> # Get messages
>> typ, data = server.search(None, 'ALL')
>> for num in data[0].split():
>> typ, data = server.fetch(num, '(RFC822)')
>> tmp = file(TMPFILE, 'w+')
>> tmp.write(data[0][1])
>> tmp.close()
>> log.write(commands.getoutput("%s --prefs-file=%s --spam %s" % \
>> (SALEARN, PREFS, TMPFILE)))
>> log.write("\n")
>> # Mark learned spam as "Deleted"
>> server.store(num, '+FLAGS', '\\Deleted')
>> # Delete messages marked as "Deleted" from server
>> server.expunge()
>> server.logout
>>
>>
>> ------------------------------------------------------------------------
>>
>> #!/usr/bin/env python
>> import commands, os, time
>> import imaplib
>> import sys, re
>> import string, random
>> import StringIO, rfc822
>>
>> # Set required variables
>> PREFS = "/opt/MailScanner/etc/spam.assassin.prefs.conf"
>> TMPFILE = "/var/tmp/salearn.tmp"
>> SALEARN = "/usr/bin/sa-learn"
>> SERVER = "x.x.x.x"
>> USER = "someuserwithaccesstopublicfolder"
>> PASSWORD = "somepassword"
>> LOGFILE = "/var/log/learn.spam.log"
>> log = file(LOGFILE, 'a+')
>> log.write("\n\nTraining SpamAssassin on %s at %s\n" % (time.strftime("%Y-%m-%d"), time.strftime("%H:%M:%S")))
>>
>> # connect to server
>> server = imaplib.IMAP4(SERVER)
>>
>> # login
>> server.login(USER, PASSWORD)
>> server.select("Public Folders/Spam")
>>
>> # Get messages
>> typ, data = server.search(None, 'ALL')
>> for num in data[0].split():
>> typ, data = server.fetch(num, '(RFC822)')
>> tmp = file(TMPFILE, 'w+')
>> tmp.write(data[0][1])
>> tmp.close()
>> log.write(commands.getoutput("%s --prefs-file=%s --spam %s" % \
>> (SALEARN, PREFS, TMPFILE)))
>> log.write("\n")
>> # Mark learned spam as "Deleted"
>> server.store(num, '+FLAGS', '\\Deleted')
>> # Delete messages marked as "Deleted" from server
>> #server.expunge()
>> server.logout
>
>
RE: sa-learn on dedicated spamabuse email account
Posted by Dan Barker <db...@visioncomm.net>.
Kris:
You mention "and Outlook is a headache to do properly but it *is* possible."
That got me thinking of how to do it. If I craft an email with no content but the FP or FN email drag-n-droped into the message body, I get a message/rfc822 Content-Type. If I send this to the spam or ham learning address, will sa-learn ignore my outer message and parse only the rfc822 part?
The way I train FP's is just forward the whole thing to the mbox file where sa-learn will train - SpamAssassin has made an rfc822 attachment out of the email as part of it's reporting. For FN's, I've not found a good way, but your comment got me to thinking about the drag-n-drop. (Drag to the desktop, and include that file does the same thing. SaveAs does NOT!).
I guess sa-learn has a -D switch, but would I be able to tell what it was doing at the level required here (which message bits are being learned)?
I may have to try that unless someone (or Kris) already knows the answer and can save me the trouble.
Dan
-----Original Message-----
From: Kris Deugau [mailto:kdeugau@vianet.ca]
Sent: Tuesday, January 30, 2007 2:24 PM
To: Oenus Tech Services
Cc: users@spamassassin.apache.org
Subject: Re: sa-learn on dedicated spamabuse email account
Oenus Tech Services wrote:
> Yes, but this would not work with our more than 1500 customers that have
> only pop3 access and do not have access to any shared or private folder
> in the servers. We needed to implement some way for these pop3-only
> customers to report spam back to us, and for now we've only thought of
> forwarding spam to an spamabuse account where some scripts could check
> its inbox and do bayesian learning. However, does sa-learn take into
> account that those emails are being forwarded
That depends on what you mean. If you just click "Forward" in most mail
clients, the result will be useless for feeding Bayes. (Mostly. The
exceptions are rare.)
However, if you spend the effort to teach people how to "Forward as
attachment" (most mail programs support this properly; Eudora is the
only one I know of where it's simply not possible - someone *please*
prove me wrong? - and Outlook is a headache to do properly but it *is*
possible.)
Hand-check the submissions for a while though, because customers *will*
misclick something (you do NOT want to train ham as spam) or submit
something they really *don't* want, but which is still ham because it's
a "Joke-A-Day"-type newsletter that many more people want - so you have
to blacklist it for that user.
Of course, if you've got the server resources for per-user Bayes -
automate away, customers can't hurt each others' mail flow. >;) You
might also want to keep submissions for a week or so in case $user calls
in complaining their favourite "Please Spam Me And My Friends"
newsletter isn't showing up in their inbox - you can dig it out and
hand-train it correctly.
-kgd
Re: sa-learn on dedicated spamabuse email account
Posted by Kris Deugau <kd...@vianet.ca>.
Oenus Tech Services wrote:
> Yes, but this would not work with our more than 1500 customers that have
> only pop3 access and do not have access to any shared or private folder
> in the servers. We needed to implement some way for these pop3-only
> customers to report spam back to us, and for now we've only thought of
> forwarding spam to an spamabuse account where some scripts could check
> its inbox and do bayesian learning. However, does sa-learn take into
> account that those emails are being forwarded
That depends on what you mean. If you just click "Forward" in most mail
clients, the result will be useless for feeding Bayes. (Mostly. The
exceptions are rare.)
However, if you spend the effort to teach people how to "Forward as
attachment" (most mail programs support this properly; Eudora is the
only one I know of where it's simply not possible - someone *please*
prove me wrong? - and Outlook is a headache to do properly but it *is*
possible.)
Hand-check the submissions for a while though, because customers *will*
misclick something (you do NOT want to train ham as spam) or submit
something they really *don't* want, but which is still ham because it's
a "Joke-A-Day"-type newsletter that many more people want - so you have
to blacklist it for that user.
Of course, if you've got the server resources for per-user Bayes -
automate away, customers can't hurt each others' mail flow. >;) You
might also want to keep submissions for a week or so in case $user calls
in complaining their favourite "Please Spam Me And My Friends"
newsletter isn't showing up in their inbox - you can dig it out and
hand-train it correctly.
-kgd
Re: sa-learn on dedicated spamabuse email account
Posted by Oenus Tech Services <oe...@oenus.com>.
Thanks, Peter
Yes, but this would not work with our more than 1500 customers that have
only pop3 access and do not have access to any shared or private folder
in the servers. We needed to implement some way for these pop3-only
customers to report spam back to us, and for now we've only thought of
forwarding spam to an spamabuse account where some scripts could check
its inbox and do bayesian learning. However, does sa-learn take into
account that those emails are being forwarded and they're not the
original source of the spam? I guess not. If that is the case, does
anybody has come up with a similar idea?
Ignacio
Peter Russell escribió:
> See attach python script written by one of the folks on the MailScanner
> list.
>
> Its designed for use with exchange, so i will describe the Exchange
> usage and you can modify as you see fit to work on your pop3 server.
>
> In Exchange 2003 create a public folder called SPAM, give everyone
> contributor access, not read or edit. Then any user can simply drag spam
> to the public folder, but no user can see in the folder.
>
> Modify the script to suit your environment (Exchange server name and
> credentials). Make it executable.
>
> Now run it. It will scan your public folder called SPAM and learn the
> contents into bayes, then delete the messages it has learned.
>
> The script doesnt seem to run recursively all that well, it maye stop
> randomly and need to be re run again - if any python scripters see this
> would you mind having a go fixing this and re posting to the list?
>
> Many thanks
> Pete
>
> Oenus Tech Services wrote:
>> Hi there!
>>
>> Most of our email is delivered through pop3, so right now bayes
>> filtering is off. Nevertheless Spamassassin is doing a good job
>> filtering email, but we want to setup a way for our customers to report
>> to us undetected spam by forwarding that spam to an
>> spamabuse@example.com account in our server. If we then point sa-learn
>> to that inbox, will it work? My concern is that email arriving to that
>> account is not from the spammer anymore, but from a forwarded mail by
>> our customer.
>>
>> TIA,
>>
>> Ignacio
>>
>
> ------------------------------------------------------------------------
>
> #!/usr/bin/env python
> import commands, os, time
> import imaplib
> import sys, re
> import string, random
> import StringIO, rfc822
>
> # Set required variables
> PREFS = "/etc/MailScanner/spam.assassin.prefs.conf"
> TMPFILE = "/var/tmp/salearn.tmp"
> SALEARN = "/usr/bin/sa-learn"
> SERVER = "x.x.x.x"
> USER = "someuserwithaccesstopublicfolder"
> PASSWORD = "somepassword"
> LOGFILE = "/var/log/learn.spam.log"
> log = file(LOGFILE, 'a+')
> log.write("\n\nTraining SpamAssassin on %s at %s\n" % (time.strftime("%Y-%m-%d"), time.strftime("%H:%M:%S")))
>
> # connect to server
> server = imaplib.IMAP4(SERVER)
>
> # login
> server.login(USER, PASSWORD)
> server.select("Public Folders/Spam")
>
> # Get messages
> typ, data = server.search(None, 'ALL')
> for num in data[0].split():
> typ, data = server.fetch(num, '(RFC822)')
> tmp = file(TMPFILE, 'w+')
> tmp.write(data[0][1])
> tmp.close()
> log.write(commands.getoutput("%s --prefs-file=%s --spam %s" % \
> (SALEARN, PREFS, TMPFILE)))
> log.write("\n")
> # Mark learned spam as "Deleted"
> server.store(num, '+FLAGS', '\\Deleted')
> # Delete messages marked as "Deleted" from server
> server.expunge()
> server.logout
>
>
> ------------------------------------------------------------------------
>
> #!/usr/bin/env python
> import commands, os, time
> import imaplib
> import sys, re
> import string, random
> import StringIO, rfc822
>
> # Set required variables
> PREFS = "/opt/MailScanner/etc/spam.assassin.prefs.conf"
> TMPFILE = "/var/tmp/salearn.tmp"
> SALEARN = "/usr/bin/sa-learn"
> SERVER = "x.x.x.x"
> USER = "someuserwithaccesstopublicfolder"
> PASSWORD = "somepassword"
> LOGFILE = "/var/log/learn.spam.log"
> log = file(LOGFILE, 'a+')
> log.write("\n\nTraining SpamAssassin on %s at %s\n" % (time.strftime("%Y-%m-%d"), time.strftime("%H:%M:%S")))
>
> # connect to server
> server = imaplib.IMAP4(SERVER)
>
> # login
> server.login(USER, PASSWORD)
> server.select("Public Folders/Spam")
>
> # Get messages
> typ, data = server.search(None, 'ALL')
> for num in data[0].split():
> typ, data = server.fetch(num, '(RFC822)')
> tmp = file(TMPFILE, 'w+')
> tmp.write(data[0][1])
> tmp.close()
> log.write(commands.getoutput("%s --prefs-file=%s --spam %s" % \
> (SALEARN, PREFS, TMPFILE)))
> log.write("\n")
> # Mark learned spam as "Deleted"
> server.store(num, '+FLAGS', '\\Deleted')
> # Delete messages marked as "Deleted" from server
> #server.expunge()
> server.logout
Re: sa-learn on dedicated spamabuse email account
Posted by Peter Russell <pe...@enitech.com.au>.
See attach python script written by one of the folks on the MailScanner
list.
Its designed for use with exchange, so i will describe the Exchange
usage and you can modify as you see fit to work on your pop3 server.
In Exchange 2003 create a public folder called SPAM, give everyone
contributor access, not read or edit. Then any user can simply drag spam
to the public folder, but no user can see in the folder.
Modify the script to suit your environment (Exchange server name and
credentials). Make it executable.
Now run it. It will scan your public folder called SPAM and learn the
contents into bayes, then delete the messages it has learned.
The script doesnt seem to run recursively all that well, it maye stop
randomly and need to be re run again - if any python scripters see this
would you mind having a go fixing this and re posting to the list?
Many thanks
Pete
Oenus Tech Services wrote:
> Hi there!
>
> Most of our email is delivered through pop3, so right now bayes
> filtering is off. Nevertheless Spamassassin is doing a good job
> filtering email, but we want to setup a way for our customers to report
> to us undetected spam by forwarding that spam to an
> spamabuse@example.com account in our server. If we then point sa-learn
> to that inbox, will it work? My concern is that email arriving to that
> account is not from the spammer anymore, but from a forwarded mail by
> our customer.
>
> TIA,
>
> Ignacio
>