You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Jacques Perrolle <pe...@radonc.duke.edu> on 2004/04/16 23:16:53 UTC
Bayes broken beyond repair
Thanks for taking the time to read this..
I have Googled myself silly and just cannot seem to come up with the
answer.
A bit about the set-up.
It's a Solaris box, and we have successfully been using Spamassassin
(and Postfix) for a year or so to help mark spam as spam.
Lately I've been creating my own rules to help it be more accurate for
our domain. This naturally led me to want to get the site-wide
bayesian filtering going. Which I have failed at, pretty
spectacularly.
All of my inabilities to get bayesian filtering going aside, I've run
into a problem that I cannot overcome. I'm guessing that I've totally
broken something and I cannot find what/where, nor can I seem to be
able to start over from scratch. At this point, I can feed it either
spam or ham, but not both. If I feed it one, it happily eats it. Then
if I try to feed the other, it breaks. Either way, if I look at the
'magic dump', there's nothing there at all. All 5000 ham/spam I just
fed it, isn't there. I'm guessing that I've missed removing something
when I was tried to start over from scratch.
I just want to start over, naturally not breaking Spamassassin. Any
suggestions?
No one wants to see a grown man cry.
Jacques
Re: Bayes broken beyond repair
Posted by Jacques Perrolle <pe...@radonc.duke.edu>.
Sheesh! How can I forget??
spamassassin -V
SpamAssassin version 2.63
Thanks again,
Have a great weekend.
Jacques
On Apr 16, 2004, at 5:16 PM, Jacques Perrolle wrote:
> Thanks for taking the time to read this..
>
> I have Googled myself silly and just cannot seem to come up with the
> answer.
>
> A bit about the set-up.
>
> It's a Solaris box, and we have successfully been using Spamassassin
> (and Postfix) for a year or so to help mark spam as spam.
> Lately I've been creating my own rules to help it be more accurate for
> our domain. This naturally led me to want to get the site-wide
> bayesian filtering going. Which I have failed at, pretty
> spectacularly.
>
> All of my inabilities to get bayesian filtering going aside, I've run
> into a problem that I cannot overcome. I'm guessing that I've totally
> broken something and I cannot find what/where, nor can I seem to be
> able to start over from scratch. At this point, I can feed it either
> spam or ham, but not both. If I feed it one, it happily eats it.
> Then if I try to feed the other, it breaks. Either way, if I look at
> the 'magic dump', there's nothing there at all. All 5000 ham/spam I
> just fed it, isn't there. I'm guessing that I've missed removing
> something when I was tried to start over from scratch.
>
> I just want to start over, naturally not breaking Spamassassin. Any
> suggestions?
>
> No one wants to see a grown man cry.
>
> Jacques
Re: Bayes broken beyond repair
Posted by Jacques Perrolle <pe...@radonc.duke.edu>.
Definitely read the documentation multiple times, and I see that now,
but I always glossed over it because I didn't think that was the
problem.
I was not correctly comprehending what I was reading looking at the
Wiki on bayes SiteWideSetup. The bayes_file_mode, well, there sure
seem to be a lot of instances where people have set thiers to 666.
Hey, nothing else was working!
Guess I should have RTFM'd closer.
On Apr 17, 2004, at 4:58 AM, David B Funk wrote:
> I can see two problems right away.
>
> Rather than spend time Googling, try reading the SA documentation,
> particularly the docs for the SpamAssassin configuration file.
>
> Your "bayes_file_mode" and "bayes_path" are wrong.
>
> Note that "bayes_path" needs to be the base name of a file, NOT
> a directory name.
>
> EG: "bayes_path /etc/mail/spamassassin/bayes/bayes-file"
>
> NOT: "bayes_path /etc/mail/spamassassin/bayes/"
> NOR: "bayes_path /etc/mail/spamassassin/bayes"
>
> where /etc/mail/spamassassin/bayes is an already existing directory.
>
> Also make sure that the user-id which spamd runs as has write
> permissions to the directory that the bayes files will be put in.
>
> I'll let you read the docs to see what you've done wrong with
> your "bayes_file_mode". ;)
>
> On Fri, 16 Apr 2004, Jacques Perrolle wrote:
>
>> Thanks for replying,
>>
>> I have indeed, but the output didn't mean a whole lot to me and I
>> couldn't come up with anything concrete Googling.
>> Mainly the reason I just wanted to start over from scratch without
>> "damaging" the SA install.
>>
>> Note the "debug: bayes: Not available for scanning, only 0 spam(s) in
>> Bayes DB < 200" even though I dumped in 4500 hams.
>>
>> Permissions according to local.cf is: "bayes_file_mode 0666"
>>
>> Thanks again.
>>
>> Hopefully no one will mind if I dump the output here:
>>
>> debug: Score set 0 chosen.
>> debug: running in taint mode? yes
>> debug: Running in taint mode, removing unsafe env vars, and resetting
> [snip..]
>
>> /home/foo:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/ccs/bin
>> debug: using "/usr/local/share/spamassassin" for default rules dir
>> debug: using "/etc/mail/spamassassin" for site rules dir
>> debug: using "/home/foo/.spamassassin/user_prefs" for user prefs file
>> debug: bayes: 21558 tie-ing to DB file R/O
>> /etc/mail/spamassassin/bayes/_toks
>> debug: bayes: 21558 tie-ing to DB file R/O
>> /etc/mail/spamassassin/bayes/_seen
>> debug: bayes: found bayes db version 0
>> debug: bayes: Not available for scanning, only 0 spam(s) in Bayes DB <
>> 200
> [snip..]
>>
>> Learned from 0 message(s) (1 message(s) examined).
>> debug: bayes: 21558 untie-ing
>> ERROR: the Bayes learn function returned an error, please re-run with
>> -D for more information
>
> --
> Dave Funk University of Iowa
> <dbfunk (at) engineering.uiowa.edu> College of Engineering
> 319/335-5751 FAX: 319/384-0549 1256 Seamans Center
> Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
> #include <std_disclaimer.h>
> Better is not better, 'standard' is better. B{
>
Re: Bayes broken beyond repair
Posted by David B Funk <db...@engineering.uiowa.edu>.
I can see two problems right away.
Rather than spend time Googling, try reading the SA documentation,
particularly the docs for the SpamAssassin configuration file.
Your "bayes_file_mode" and "bayes_path" are wrong.
Note that "bayes_path" needs to be the base name of a file, NOT
a directory name.
EG: "bayes_path /etc/mail/spamassassin/bayes/bayes-file"
NOT: "bayes_path /etc/mail/spamassassin/bayes/"
NOR: "bayes_path /etc/mail/spamassassin/bayes"
where /etc/mail/spamassassin/bayes is an already existing directory.
Also make sure that the user-id which spamd runs as has write
permissions to the directory that the bayes files will be put in.
I'll let you read the docs to see what you've done wrong with
your "bayes_file_mode". ;)
On Fri, 16 Apr 2004, Jacques Perrolle wrote:
> Thanks for replying,
>
> I have indeed, but the output didn't mean a whole lot to me and I
> couldn't come up with anything concrete Googling.
> Mainly the reason I just wanted to start over from scratch without
> "damaging" the SA install.
>
> Note the "debug: bayes: Not available for scanning, only 0 spam(s) in
> Bayes DB < 200" even though I dumped in 4500 hams.
>
> Permissions according to local.cf is: "bayes_file_mode 0666"
>
> Thanks again.
>
> Hopefully no one will mind if I dump the output here:
>
> debug: Score set 0 chosen.
> debug: running in taint mode? yes
> debug: Running in taint mode, removing unsafe env vars, and resetting
[snip..]
> /home/foo:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/ccs/bin
> debug: using "/usr/local/share/spamassassin" for default rules dir
> debug: using "/etc/mail/spamassassin" for site rules dir
> debug: using "/home/foo/.spamassassin/user_prefs" for user prefs file
> debug: bayes: 21558 tie-ing to DB file R/O
> /etc/mail/spamassassin/bayes/_toks
> debug: bayes: 21558 tie-ing to DB file R/O
> /etc/mail/spamassassin/bayes/_seen
> debug: bayes: found bayes db version 0
> debug: bayes: Not available for scanning, only 0 spam(s) in Bayes DB <
> 200
[snip..]
>
> Learned from 0 message(s) (1 message(s) examined).
> debug: bayes: 21558 untie-ing
> ERROR: the Bayes learn function returned an error, please re-run with
> -D for more information
--
Dave Funk University of Iowa
<dbfunk (at) engineering.uiowa.edu> College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{
Re: Bayes broken beyond repair
Posted by Jacques Perrolle <pe...@radonc.duke.edu>.
Thanks for replying,
I have indeed, but the output didn't mean a whole lot to me and I
couldn't come up with anything concrete Googling.
Mainly the reason I just wanted to start over from scratch without
"damaging" the SA install.
Note the "debug: bayes: Not available for scanning, only 0 spam(s) in
Bayes DB < 200" even though I dumped in 4500 hams.
Permissions according to local.cf is: "bayes_file_mode 0666"
Thanks again.
Hopefully no one will mind if I dump the output here:
debug: Score set 0 chosen.
debug: running in taint mode? yes
debug: Running in taint mode, removing unsafe env vars, and resetting
PATH
debug: PATH included '/home/foo', keeping.
debug: PATH included '/usr/bin', keeping.
debug: PATH included '/bin', keeping.
debug: PATH included '/usr/sbin', keeping.
debug: PATH included '/sbin', keeping.
debug: PATH included '/usr/local/bin', keeping.
debug: PATH included '/usr/ccs/bin', keeping.
debug: Final PATH set to:
/home/foo:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/ccs/bin
debug: using "/usr/local/share/spamassassin" for default rules dir
debug: using "/etc/mail/spamassassin" for site rules dir
debug: using "/home/foo/.spamassassin/user_prefs" for user prefs file
debug: bayes: 21558 tie-ing to DB file R/O
/etc/mail/spamassassin/bayes/_toks
debug: bayes: 21558 tie-ing to DB file R/O
/etc/mail/spamassassin/bayes/_seen
debug: bayes: found bayes db version 0
debug: bayes: Not available for scanning, only 0 spam(s) in Bayes DB <
200
debug: bayes: 21558 untie-ing
debug: bayes: 21558 untie-ing db_toks
debug: bayes: 21558 untie-ing db_seen
debug: Score set 0 chosen.
debug: Initialising learner
debug: Initialising learner
debug: Syncing Bayes journal and expiring old tokens...
debug: lock: 21558 created
/etc/mail/spamassassin/bayes/.lock.foohostname
debug: lock: 21558 trying to get lock on /etc/mail/spamassassin/bayes/
with 0 retries
debug: lock: 21558 link to /etc/mail/spamassassin/bayes/.lock: link ok
debug: bayes: 21558 tie-ing to DB file R/W
/etc/mail/spamassassin/bayes/_toks
debug: bayes: 21558 tie-ing to DB file R/W
/etc/mail/spamassassin/bayes/_seen
debug: bayes: found bayes db version 0
debug: bayes: detected bayes db format 0, upgrading
debug: bayes: upgrading database format from v0 to v2
debug: bayes: 21558 untie-ing
debug: bayes: 21558 untie-ing db_toks
debug: bayes: 21558 untie-ing db_seen
debug: bayes: files locked, now unlocking lock
debug: unlock: 21558 unlink /etc/mail/spamassassin/bayes/.lock
debug: Syncing complete.
debug: Removing Markup
debug: Learning Spam
debug: uri tests: Done uriRE
debug: lock: 21558 created
/etc/mail/spamassassin/bayes/.lock.foohostname
debug: lock: 21558 trying to get lock on /etc/mail/spamassassin/bayes/
with 0 retries
debug: lock: 21558 link to /etc/mail/spamassassin/bayes/.lock: link ok
debug: bayes: 21558 tie-ing to DB file R/W
/etc/mail/spamassassin/bayes/_toks
debug: bayes: 21558 tie-ing to DB file R/W
/etc/mail/spamassassin/bayes/_seen
debug: bayes: found bayes db version 0
debug: bayes: detected bayes db format 0, upgrading
debug: bayes: upgrading database format from v0 to v2
debug: bayes: 21558 untie-ing
debug: bayes: 21558 untie-ing db_toks
debug: bayes: 21558 untie-ing db_seen
debug: bayes: files locked, now unlocking lock
debug: unlock: 21558 unlink /etc/mail/spamassassin/bayes/.lock
Learned from 0 message(s) (1 message(s) examined).
debug: bayes: 21558 untie-ing
ERROR: the Bayes learn function returned an error, please re-run with
-D for more information
On Apr 16, 2004, at 5:27 PM, Ryan Moore wrote:
>
> Have you tried 'sa-learn -D --[ham|spam]' to see if you get any errors
> there? Also verify the permissions for the bayes path are sane.
>
>
> Ryan Moore
> ----------
> Perigee.net Corporation
> 704-849-8355 (sales)
> 704-849-8017 (tech)
> www.perigee.net
Re: Bayes broken beyond repair
Posted by Jacques Perrolle <pe...@radonc.duke.edu>.
Nope, been using that. But I appreciate any help I can get!
Thanks!
On Apr 16, 2004, at 6:59 PM, Dan MacNeil wrote:
>
>>> dump', there's nothing there at all. All 5000 ham/spam I just fed
>>> it,
>
> I had a similar problem until I realized I needed the --mbox switch to
> sa-learn when feeding a single mbox file
>
> [hope this is relivant, coming in on the middle]
>
Re: Bayes broken beyond repair
Posted by Dan MacNeil <om...@brave.cs.uml.edu>.
> > dump', there's nothing there at all. All 5000 ham/spam I just fed it,
I had a similar problem until I realized I needed the --mbox switch to
sa-learn when feeding a single mbox file
[hope this is relivant, coming in on the middle]
> Jacques Perrolle wrote:
> > All of my inabilities to get bayesian filtering going aside, I've run
> > into a problem that I cannot overcome. I'm guessing that I've totally
> > broken something and I cannot find what/where, nor can I seem to be able
> > to start over from scratch. At this point, I can feed it either spam or
> > ham, but not both. If I feed it one, it happily eats it. Then if I try
> > to feed the other, it breaks. Either way, if I look at the 'magic
> > dump', there's nothing there at all. All 5000 ham/spam I just fed it,
> > isn't there. I'm guessing that I've missed removing something when I
> > was tried to start over from scratch.
> >
> > I just want to start over, naturally not breaking Spamassassin. Any
> > suggestions?
> >
> > No one wants to see a grown man cry.
> >
> > Jacques
> >
>
> Have you tried 'sa-learn -D --[ham|spam]' to see if you get any errors
> there? Also verify the permissions for the bayes path are sane.
>
>
> Ryan Moore
> ----------
> Perigee.net Corporation
> 704-849-8355 (sales)
> 704-849-8017 (tech)
> www.perigee.net
>
>
Re: Bayes broken beyond repair
Posted by Ryan Moore <ry...@perigee.net>.
Jacques Perrolle wrote:
> All of my inabilities to get bayesian filtering going aside, I've run
> into a problem that I cannot overcome. I'm guessing that I've totally
> broken something and I cannot find what/where, nor can I seem to be able
> to start over from scratch. At this point, I can feed it either spam or
> ham, but not both. If I feed it one, it happily eats it. Then if I try
> to feed the other, it breaks. Either way, if I look at the 'magic
> dump', there's nothing there at all. All 5000 ham/spam I just fed it,
> isn't there. I'm guessing that I've missed removing something when I
> was tried to start over from scratch.
>
> I just want to start over, naturally not breaking Spamassassin. Any
> suggestions?
>
> No one wants to see a grown man cry.
>
> Jacques
>
Have you tried 'sa-learn -D --[ham|spam]' to see if you get any errors
there? Also verify the permissions for the bayes path are sane.
Ryan Moore
----------
Perigee.net Corporation
704-849-8355 (sales)
704-849-8017 (tech)
www.perigee.net