You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Steven Stern <su...@sterndata.com> on 2006/05/10 14:24:16 UTC

Bayes not working

On a new SA installation that's as identical to the other 3 we have
running as possible, bayes is not running.

spamassassin -D --lint indicates that all is normal. The test message
generates a Bayes score.  sa-learn is able to talk to the mysql
database:  We're able to update the database using sa-learn.

However, in production, spamassassin does not report any BAYES_ scores.
 When the spam value exceeds the threshold that would normally cause
autolearning, "autolearn=no" changes to "autolearn=unavailable".
Similarly, AWL entries are not being created.

Can anyone see what's wrong?

[3320] dbg: config: read file /usr/share/spamassassin/23_bayes.cf
[3320] dbg: bayes: using username: root
[3320] dbg: bayes: database connection established
[3320] dbg: bayes: found bayes db version 3
[3320] dbg: bayes: Using userid: 1
[3320] dbg: bayes: corpus size: nspam = 178, nham = 168
[3320] dbg: bayes: tok_get_all: token count: 20
[3320] dbg: bayes: score = 0.913557143318889
[3320] dbg: rules: ran eval rule BAYES_80 ======> got hit
[3320] dbg: auto-whitelist: sql-based connected to
DBI:mysql:sa_bayes:ccim-mx2
[3320] dbg: auto-whitelist: sql-based finish: disconnected from
DBI:mysql:sa_bayes:ccim-mx2
[3320] dbg: check:
tests=BAYES_80,MISSING_SUBJECT,NO_REAL_NAME,NO_RECEIVED,NO_RELAYS,TO_CC_NONE



# grep -i bayes local.cf
# Enable the Bayes system
use_bayes               1
# Enable Bayes auto-learning
bayes_auto_learn        1
bayes_min_ham_num       100
bayes_min_spam_num      100
# bayes_path            /var/spool/spamassassin/bayes
bayes_store_module      Mail::SpamAssassin::BayesStore::MySQL
bayes_sql_dsn           DBI:mysql:sa_bayes:ccim-mx2
bayes_sql_username      spamass
bayes_sql_password      xxx
bayes_sql_override_username     root
bayes_auto_expire       0
user_awl_dsn                    DBI:mysql:sa_bayes:ccim-mx2

# grep -i awl local.cf
user_awl_dsn                    DBI:mysql:sa_bayes:ccim-mx2
user_awl_sql_table           awl
user_awl_sql_username           spamass
user_awl_sql_password           xxx
user_awl_sql_override_username  root


]# ps -ef |grep spam
root      2170     1  0 07:01 ?        00:00:04 /usr/bin/spamd -d -c -m5
-H -r /var/run/spamd.pid
root      2247  2170  1 07:01 ?        00:00:20 spamd child
root      2248  2170  0 07:01 ?        00:00:00 spamd child
sa-milt   3264     1  0 07:15 pts/0    00:00:00 /bin/bash
/usr/sbin/spamass-milter-wrapper -p
/var/run/spamass-milter/spamass-milter.sock -P
/var/run/spamass-milter.pid -i 127.0.0.1 -r 10 -- -d localhost -p 783
sa-milt   3265  3264  0 07:15 pts/0    00:00:00 /usr/sbin/spamass-milter
-p /var/run/spamass-milter/spamass-milter.sock -P
/var/run/spamass-milter.pid -i 127.0.0.1 -r 10 -- -d localhost -p 783


SpamAssassin version 3.1.1
  running on Perl version 5.8.6
spamass-milter - Version 0.3.1


-- 

  Steve

Re: Bayes not working

Posted by Matt Kettler <mk...@evi-inc.com>.
Steven Stern wrote:
>>   
> Everything's tweaked to use "root" as the user. We do sitewide
> processing since this sits on an MX server.
> 

Do you use spamd? If so, it WILL NOT use root as the user. Ever. Period.

Re: Bayes not working

Posted by Steven Stern <su...@sterndata.com>.
Michael Monnerie wrote:
> On Mittwoch, 10. Mai 2006 16:01 Steven Stern wrote:
>   
>> I imported a corpus of about 20000 messages total and it wasn't
>> working. I blew it all away and started from scratch thinking that
>> was the problem.  For now, local.cf has a minimum of 100 messages of
>> each type. The current database exceeds that.
>>     
>
> I've had such an issue. In ancient times I had done "sudo -H -u 
> spamscanner sa-learn ....", but that doesn't work now. I really have to 
> do "su -l spamscanner" and then "sa-learn". Maybe that's your problem.
>
> Try to "sa-learn --dump magic|grep token" to see how many ham/spam there 
> really are  - as that user.
>
>
>   
Everything's tweaked to use "root" as the user. We do sitewide 
processing since this sits on an MX server.


Re: Bayes not working

Posted by Michael Monnerie <mi...@it-management.at>.
On Mittwoch, 10. Mai 2006 16:01 Steven Stern wrote:
> I imported a corpus of about 20000 messages total and it wasn't
> working. I blew it all away and started from scratch thinking that
> was the problem.  For now, local.cf has a minimum of 100 messages of
> each type. The current database exceeds that.

I've had such an issue. In ancient times I had done "sudo -H -u 
spamscanner sa-learn ....", but that doesn't work now. I really have to 
do "su -l spamscanner" and then "sa-learn". Maybe that's your problem.

Try to "sa-learn --dump magic|grep token" to see how many ham/spam there 
really are  - as that user.

mfg zmi
-- 
// Michael Monnerie, Ing.BSc    -----      http://it-management.at
// Tel: 0660/4156531                          .network.your.ideas.
// PGP Key:   "lynx -source http://zmi.at/zmi3.asc | gpg --import"
// Fingerprint: 44A3 C1EC B71E C71A B4C2  9AA6 C818 847C 55CB A4EE
// Keyserver: www.keyserver.net                 Key-ID: 0x55CBA4EE

Re: Bayes not working

Posted by Steven Stern <su...@sterndata.com>.
Andy Spiegl wrote:
>> [3320] dbg: bayes: corpus size: nspam = 178, nham = 168
>>     
> Probably because your corpus is still too small.
>
> man Mail::SpamAssassin::Conf
> ...
>        bayes_min_ham_num        (Default: 200)
>        bayes_min_spam_num       (Default: 200)
>            To be accurate, the Bayes system does not activate until a
>            certain number of ham (non-spam) and spam have been learned.
>            The default is 200 of each ham and spam, but you can tune these
>            up or down with these two settings.
>   
I imported a corpus of about 20000 messages total and it wasn't working. 
I blew it all away and started from scratch thinking that was the 
problem.  For now, local.cf has a minimum of 100 messages of each type. 
The current database exceeds that.

Re: Bayes not working

Posted by Andy Spiegl <sp...@spiegl.de>.
> [3320] dbg: bayes: corpus size: nspam = 178, nham = 168
Probably because your corpus is still too small.

man Mail::SpamAssassin::Conf
...
       bayes_min_ham_num        (Default: 200)
       bayes_min_spam_num       (Default: 200)
           To be accurate, the Bayes system does not activate until a
           certain number of ham (non-spam) and spam have been learned.
           The default is 200 of each ham and spam, but you can tune these
           up or down with these two settings.
...

Bye,
 Andy.

-- 
 Finagle's Sixth Law: Don't believe in miracles -- rely on them.

RE: Bayes not working

Posted by Greg Allen <sa...@floridacpu.com>.
You will probably get more ideas from posters, but here is my thought.

Are you running spamassassin -D --lint as the user that SA runs under when
it is running live?

For instance, I call SA with the user "filter", not the user "root".

So, to properly test SA I have to first type: su filter

This makes me a superuser "filter".

Now I get a real test of SA when I run spamassassin -D --lint

It looks like you may be testing with the user "root"?

SA really should not run live under the root user.

Well, that's my idea.

Good luck




> -----Original Message-----
> From: Steven Stern [mailto:subscribed-lists@sterndata.com]
> Sent: Wednesday, May 10, 2006 8:24 AM
> To: Spamass
> Subject: Bayes not working
>
>
> On a new SA installation that's as identical to the other 3 we have
> running as possible, bayes is not running.
>
> spamassassin -D --lint indicates that all is normal. The test message
> generates a Bayes score.  sa-learn is able to talk to the mysql
> database:  We're able to update the database using sa-learn.
>
> However, in production, spamassassin does not report any BAYES_ scores.
>  When the spam value exceeds the threshold that would normally cause
> autolearning, "autolearn=no" changes to "autolearn=unavailable".
> Similarly, AWL entries are not being created.
>
> Can anyone see what's wrong?
>
> [3320] dbg: config: read file /usr/share/spamassassin/23_bayes.cf
> [3320] dbg: bayes: using username: root
> [3320] dbg: bayes: database connection established
> [3320] dbg: bayes: found bayes db version 3
> [3320] dbg: bayes: Using userid: 1
> [3320] dbg: bayes: corpus size: nspam = 178, nham = 168
> [3320] dbg: bayes: tok_get_all: token count: 20
> [3320] dbg: bayes: score = 0.913557143318889
> [3320] dbg: rules: ran eval rule BAYES_80 ======> got hit
> [3320] dbg: auto-whitelist: sql-based connected to
> DBI:mysql:sa_bayes:ccim-mx2
> [3320] dbg: auto-whitelist: sql-based finish: disconnected from
> DBI:mysql:sa_bayes:ccim-mx2
> [3320] dbg: check:
> tests=BAYES_80,MISSING_SUBJECT,NO_REAL_NAME,NO_RECEIVED,NO_RELAYS,
> TO_CC_NONE
>
>
>
> # grep -i bayes local.cf
> # Enable the Bayes system
> use_bayes               1
> # Enable Bayes auto-learning
> bayes_auto_learn        1
> bayes_min_ham_num       100
> bayes_min_spam_num      100
> # bayes_path            /var/spool/spamassassin/bayes
> bayes_store_module      Mail::SpamAssassin::BayesStore::MySQL
> bayes_sql_dsn           DBI:mysql:sa_bayes:ccim-mx2
> bayes_sql_username      spamass
> bayes_sql_password      xxx
> bayes_sql_override_username     root
> bayes_auto_expire       0
> user_awl_dsn                    DBI:mysql:sa_bayes:ccim-mx2
>
> # grep -i awl local.cf
> user_awl_dsn                    DBI:mysql:sa_bayes:ccim-mx2
> user_awl_sql_table           awl
> user_awl_sql_username           spamass
> user_awl_sql_password           xxx
> user_awl_sql_override_username  root
>
>
> ]# ps -ef |grep spam
> root      2170     1  0 07:01 ?        00:00:04 /usr/bin/spamd -d -c -m5
> -H -r /var/run/spamd.pid
> root      2247  2170  1 07:01 ?        00:00:20 spamd child
> root      2248  2170  0 07:01 ?        00:00:00 spamd child
> sa-milt   3264     1  0 07:15 pts/0    00:00:00 /bin/bash
> /usr/sbin/spamass-milter-wrapper -p
> /var/run/spamass-milter/spamass-milter.sock -P
> /var/run/spamass-milter.pid -i 127.0.0.1 -r 10 -- -d localhost -p 783
> sa-milt   3265  3264  0 07:15 pts/0    00:00:00 /usr/sbin/spamass-milter
> -p /var/run/spamass-milter/spamass-milter.sock -P
> /var/run/spamass-milter.pid -i 127.0.0.1 -r 10 -- -d localhost -p 783
>
>
> SpamAssassin version 3.1.1
>   running on Perl version 5.8.6
> spamass-milter - Version 0.3.1
>
>
> --
>
>   Steve
>
>