You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Jerry Malcolm <te...@malcolms.com> on 2019/11/24 18:35:16 UTC
Bayes
Thanks again to everyone who helped me get everything up and running
over the past couple of days.
Now that I have SA finding my bayes database, I'm curious about bayes
reporting. My bayes db is not new. I migrated my previous bayes db
from my old installation. And I've been training it with ~100+ new spam
messages in the past day. But I'm concerned that I'm not seeing
anything related to bayes in any scoring report. Nothing on any spam
message or good message over the past hundreds of message that have
arrived since getting bayes up and running in SA. I would expect at
least some messages to have a bayes rule fire, either positively or
negatively, or even just a 0.0 score based on bayes analysis. Is it
possible that after all of the setup I've done that bayes is still not
activated? Is there a way to force a bayes entry in the scoring summary
for an email just to confirm that it is indeed part of the scoring
process for spamd? I added "use_bayes 1" to the config just in case.
But no difference. I've got allow_tell activated, and training with an
email is giving me a good rc (It was correctly giving an error before I
added allow_tell). Just nothing in the scoring results for any email.
Re: Bayes
Posted by Jerry Malcolm <te...@malcolms.com>.
Can I bump this one to the top again? I had great bayes reports in
every email for 30 minutes. Then nothing for the last three days, even
after restarting SA. Is it possible that my bayes db got corrupted?
The sa-learn --dump magic looks ok as far as I can tell.
Thanks for any suggestions.
Jerry
On 11/26/2019 11:15 PM, Jerry Malcolm wrote:
>
> This is getting stranger by the minute... After playing around and
> verifying permissions and everything, I actually started getting a
> bayes score item for each email. So I celebrated and went to dinner.
> Came back a few hours later and checked the logs. Bayes consistently
> added a score line to every email for exactly 30 minutes of run time.
> Then for the last ~5 hours.... nothing. It's like the bayes component
> crashed, but SA kept running otherwise. Mail is still being processed
> via SA and being scored as usual. But bayes hasn't participated in
> any mail for ~5 hours. I'm running spamd with -D option. There was a
> lot of bayes activity in the log while it was participating. Now
> periodically I get "tie-ing to DB file R/O
> /home/spamd/bayes/bayes_toks" and "found bayes db version 3" in the
> log. That's still occurring. But nothing else related to bayes in
> the log after the first 30 minutes.
>
> Anybody have any idea about why it would work for 30 minutes, then
> just bypass from then on?
>
> Jerry
>
> On 11/26/2019 6:29 PM, Jerry Malcolm wrote:
>> On 11/25/2019 3:02 PM, Mikael Syska wrote:
>>> Try and run:
>>> sa-learn --dump magic
>>>
>>> Should give you some information like:
>>> 0.000 0 3 0 non-token data: bayes db version
>>> 0.000 0 493422 0 non-token data: nspam
>>> 0.000 0 3867414 0 non-token data: nham
>>> 0.000 0 937781 0 non-token data: ntokens
>>> 0.000 0 1573870989 0 non-token data: oldest atime
>>> 0.000 0 1574715499 0 non-token data: newest atime
>>> 0.000 0 0 0 non-token data: last journal
>>> sync atime
>>> 0.000 0 1574712064 0 non-token data: last expiry
>>> atime
>>> 0.000 0 812640 0 non-token data: last expire
>>> atime delta
>>> 0.000 0 204342 0 non-token data: last expire
>>> reduction count
>> Thank you so much for the suggestion. I tried running it as user
>> spamd: sa-learn -u spamd --dump magic
>>
>> I got this response: ERROR: Bayes dump returned an error, please
>> re-run with -D for more information. I ran it with -D and got the
>> following relevant lines:
>>
>> Nov 27 00:13:23.325 [17354] dbg: bayes: no dbs present, cannot tie DB
>> R/O: /home/spamd/bayes/bayes_toks
>> Nov 27 00:13:23.326 [17354] dbg: bayes: no dbs present, cannot tie DB
>> R/O: /home/spamd/bayes/bayes_toks
>>
>> Just for fun, I tried sudo su and reran it, and got the output
>> below. So for whatever reason, either I'm not specifying to run
>> sa-learn as spamd user correctly or user spamd appears to not have
>> permissions to its own home dir. I did check that the bayes folder
>> and all bayes_* files inside it were owned by spamd. Any ideas? BTW
>> the /var/log/maillog shows that spamd itself is indeed finding and
>> using the db in that directory.
>>
>> Thx
>>
>> Here's the sudo su output:
>>
>> [root@ip-172-31-47-84 ec2-user]# sa-learn --dump magic
>> 0.000 0 3 0 non-token data: bayes db version
>> 0.000 0 51391 0 non-token data: nspam
>> 0.000 0 6813 0 non-token data: nham
>> 0.000 0 134303 0 non-token data: ntokens
>> 0.000 0 1566344099 0 non-token data: oldest atime
>> 0.000 0 1568249462 0 non-token data: newest atime
>> 0.000 0 0 0 non-token data: last journal
>> sync atime
>> 0.000 0 0 0 non-token data: last expiry atime
>> 0.000 0 0 0 non-token data: last expire
>> atime delta
>> 0.000 0 0 0 non-token data: last expire
>> reduction count
>>>
>>> Do note that every user has it's own DB ... can be overriden with
>>> bayes_sql_override_username bayes
>>>
>>> So if either "spamd" user or your own runs it ... you should get
>>> the same result ...
>>>
>>> ( I'm actually running it in PgSQL, can't remember what the file
>>> based is called )
>>>
>>> mvh
>>> Mikael Syska
>>>
>>>
>>> ---- On Sun, 24 Nov 2019 19:35:16 +0100 *Jerry Malcolm
>>> <te...@malcolms.com>* wrote ----
>>>
>>> Thanks again to everyone who helped me get everything up and
>>> running
>>> over the past couple of days.
>>>
>>> Now that I have SA finding my bayes database, I'm curious about
>>> bayes
>>> reporting. My bayes db is not new. I migrated my previous
>>> bayes db
>>> from my old installation. And I've been training it with ~100+
>>> new spam
>>> messages in the past day. But I'm concerned that I'm not seeing
>>> anything related to bayes in any scoring report. Nothing on any
>>> spam
>>> message or good message over the past hundreds of message that have
>>> arrived since getting bayes up and running in SA. I would
>>> expect at
>>> least some messages to have a bayes rule fire, either positively or
>>> negatively, or even just a 0.0 score based on bayes analysis.
>>> Is it
>>> possible that after all of the setup I've done that bayes is
>>> still not
>>> activated? Is there a way to force a bayes entry in the scoring
>>> summary
>>> for an email just to confirm that it is indeed part of the scoring
>>> process for spamd? I added "use_bayes 1" to the config just in
>>> case.
>>> But no difference. I've got allow_tell activated, and training
>>> with an
>>> email is giving me a good rc (It was correctly giving an error
>>> before I
>>> added allow_tell). Just nothing in the scoring results for any
>>> email.
>>>
>>>
>>>
Re: Bayes
Posted by Jerry Malcolm <te...@malcolms.com>.
This is getting stranger by the minute... After playing around and
verifying permissions and everything, I actually started getting a bayes
score item for each email. So I celebrated and went to dinner. Came
back a few hours later and checked the logs. Bayes consistently added a
score line to every email for exactly 30 minutes of run time. Then for
the last ~5 hours.... nothing. It's like the bayes component crashed,
but SA kept running otherwise. Mail is still being processed via SA and
being scored as usual. But bayes hasn't participated in any mail for ~5
hours. I'm running spamd with -D option. There was a lot of bayes
activity in the log while it was participating. Now periodically I get
"tie-ing to DB file R/O /home/spamd/bayes/bayes_toks" and "found bayes
db version 3" in the log. That's still occurring. But nothing else
related to bayes in the log after the first 30 minutes.
Anybody have any idea about why it would work for 30 minutes, then just
bypass from then on?
Jerry
On 11/26/2019 6:29 PM, Jerry Malcolm wrote:
> On 11/25/2019 3:02 PM, Mikael Syska wrote:
>> Try and run:
>> sa-learn --dump magic
>>
>> Should give you some information like:
>> 0.000 0 3 0 non-token data: bayes db version
>> 0.000 0 493422 0 non-token data: nspam
>> 0.000 0 3867414 0 non-token data: nham
>> 0.000 0 937781 0 non-token data: ntokens
>> 0.000 0 1573870989 0 non-token data: oldest atime
>> 0.000 0 1574715499 0 non-token data: newest atime
>> 0.000 0 0 0 non-token data: last journal
>> sync atime
>> 0.000 0 1574712064 0 non-token data: last expiry atime
>> 0.000 0 812640 0 non-token data: last expire
>> atime delta
>> 0.000 0 204342 0 non-token data: last expire
>> reduction count
> Thank you so much for the suggestion. I tried running it as user
> spamd: sa-learn -u spamd --dump magic
>
> I got this response: ERROR: Bayes dump returned an error, please
> re-run with -D for more information. I ran it with -D and got the
> following relevant lines:
>
> Nov 27 00:13:23.325 [17354] dbg: bayes: no dbs present, cannot tie DB
> R/O: /home/spamd/bayes/bayes_toks
> Nov 27 00:13:23.326 [17354] dbg: bayes: no dbs present, cannot tie DB
> R/O: /home/spamd/bayes/bayes_toks
>
> Just for fun, I tried sudo su and reran it, and got the output
> below. So for whatever reason, either I'm not specifying to run
> sa-learn as spamd user correctly or user spamd appears to not have
> permissions to its own home dir. I did check that the bayes folder
> and all bayes_* files inside it were owned by spamd. Any ideas? BTW
> the /var/log/maillog shows that spamd itself is indeed finding and
> using the db in that directory.
>
> Thx
>
> Here's the sudo su output:
>
> [root@ip-172-31-47-84 ec2-user]# sa-learn --dump magic
> 0.000 0 3 0 non-token data: bayes db version
> 0.000 0 51391 0 non-token data: nspam
> 0.000 0 6813 0 non-token data: nham
> 0.000 0 134303 0 non-token data: ntokens
> 0.000 0 1566344099 0 non-token data: oldest atime
> 0.000 0 1568249462 0 non-token data: newest atime
> 0.000 0 0 0 non-token data: last journal
> sync atime
> 0.000 0 0 0 non-token data: last expiry atime
> 0.000 0 0 0 non-token data: last expire
> atime delta
> 0.000 0 0 0 non-token data: last expire
> reduction count
>>
>> Do note that every user has it's own DB ... can be overriden with
>> bayes_sql_override_username bayes
>>
>> So if either "spamd" user or your own runs it ... you should get the
>> same result ...
>>
>> ( I'm actually running it in PgSQL, can't remember what the file
>> based is called )
>>
>> mvh
>> Mikael Syska
>>
>>
>> ---- On Sun, 24 Nov 2019 19:35:16 +0100 *Jerry Malcolm
>> <te...@malcolms.com>* wrote ----
>>
>> Thanks again to everyone who helped me get everything up and running
>> over the past couple of days.
>>
>> Now that I have SA finding my bayes database, I'm curious about
>> bayes
>> reporting. My bayes db is not new. I migrated my previous bayes db
>> from my old installation. And I've been training it with ~100+
>> new spam
>> messages in the past day. But I'm concerned that I'm not seeing
>> anything related to bayes in any scoring report. Nothing on any spam
>> message or good message over the past hundreds of message that have
>> arrived since getting bayes up and running in SA. I would expect at
>> least some messages to have a bayes rule fire, either positively or
>> negatively, or even just a 0.0 score based on bayes analysis. Is it
>> possible that after all of the setup I've done that bayes is
>> still not
>> activated? Is there a way to force a bayes entry in the scoring
>> summary
>> for an email just to confirm that it is indeed part of the scoring
>> process for spamd? I added "use_bayes 1" to the config just in case.
>> But no difference. I've got allow_tell activated, and training
>> with an
>> email is giving me a good rc (It was correctly giving an error
>> before I
>> added allow_tell). Just nothing in the scoring results for any
>> email.
>>
>>
>>
Re: Bayes
Posted by Jerry Malcolm <te...@malcolms.com>.
On 11/25/2019 3:02 PM, Mikael Syska wrote:
> Try and run:
> sa-learn --dump magic
>
> Should give you some information like:
> 0.000 0 3 0 non-token data: bayes db version
> 0.000 0 493422 0 non-token data: nspam
> 0.000 0 3867414 0 non-token data: nham
> 0.000 0 937781 0 non-token data: ntokens
> 0.000 0 1573870989 0 non-token data: oldest atime
> 0.000 0 1574715499 0 non-token data: newest atime
> 0.000 0 0 0 non-token data: last journal
> sync atime
> 0.000 0 1574712064 0 non-token data: last expiry atime
> 0.000 0 812640 0 non-token data: last expire
> atime delta
> 0.000 0 204342 0 non-token data: last expire
> reduction count
Thank you so much for the suggestion. I tried running it as user
spamd: sa-learn -u spamd --dump magic
I got this response: ERROR: Bayes dump returned an error, please re-run
with -D for more information. I ran it with -D and got the following
relevant lines:
Nov 27 00:13:23.325 [17354] dbg: bayes: no dbs present, cannot tie DB
R/O: /home/spamd/bayes/bayes_toks
Nov 27 00:13:23.326 [17354] dbg: bayes: no dbs present, cannot tie DB
R/O: /home/spamd/bayes/bayes_toks
Just for fun, I tried sudo su and reran it, and got the output below.
So for whatever reason, either I'm not specifying to run sa-learn as
spamd user correctly or user spamd appears to not have permissions to
its own home dir. I did check that the bayes folder and all bayes_*
files inside it were owned by spamd. Any ideas? BTW the
/var/log/maillog shows that spamd itself is indeed finding and using the
db in that directory.
Thx
Here's the sudo su output:
[root@ip-172-31-47-84 ec2-user]# sa-learn --dump magic
0.000 0 3 0 non-token data: bayes db version
0.000 0 51391 0 non-token data: nspam
0.000 0 6813 0 non-token data: nham
0.000 0 134303 0 non-token data: ntokens
0.000 0 1566344099 0 non-token data: oldest atime
0.000 0 1568249462 0 non-token data: newest atime
0.000 0 0 0 non-token data: last journal
sync atime
0.000 0 0 0 non-token data: last expiry atime
0.000 0 0 0 non-token data: last expire
atime delta
0.000 0 0 0 non-token data: last expire
reduction count
>
> Do note that every user has it's own DB ... can be overriden with
> bayes_sql_override_username bayes
>
> So if either "spamd" user or your own runs it ... you should get the
> same result ...
>
> ( I'm actually running it in PgSQL, can't remember what the file based
> is called )
>
> mvh
> Mikael Syska
>
>
> ---- On Sun, 24 Nov 2019 19:35:16 +0100 *Jerry Malcolm
> <te...@malcolms.com>* wrote ----
>
> Thanks again to everyone who helped me get everything up and running
> over the past couple of days.
>
> Now that I have SA finding my bayes database, I'm curious about bayes
> reporting. My bayes db is not new. I migrated my previous bayes db
> from my old installation. And I've been training it with ~100+
> new spam
> messages in the past day. But I'm concerned that I'm not seeing
> anything related to bayes in any scoring report. Nothing on any spam
> message or good message over the past hundreds of message that have
> arrived since getting bayes up and running in SA. I would expect at
> least some messages to have a bayes rule fire, either positively or
> negatively, or even just a 0.0 score based on bayes analysis. Is it
> possible that after all of the setup I've done that bayes is still
> not
> activated? Is there a way to force a bayes entry in the scoring
> summary
> for an email just to confirm that it is indeed part of the scoring
> process for spamd? I added "use_bayes 1" to the config just in case.
> But no difference. I've got allow_tell activated, and training
> with an
> email is giving me a good rc (It was correctly giving an error
> before I
> added allow_tell). Just nothing in the scoring results for any
> email.
>
>
>
Re: Bayes
Posted by RW <rw...@googlemail.com>.
On Mon, 25 Nov 2019 22:02:03 +0100
Mikael Syska wrote:
> Try and run:
>
> sa-learn --dump magic
That needs to be run as the spamd user (using su or otherwise) for db
files.
Re: Bayes
Posted by Mikael Syska <mi...@syska.dk>.
Try and run:
sa-learn --dump magic
Should give you some information like:
0.000 0 3 0 non-token data: bayes db version
0.000 0 493422 0 non-token data: nspam
0.000 0 3867414 0 non-token data: nham
0.000 0 937781 0 non-token data: ntokens
0.000 0 1573870989 0 non-token data: oldest atime
0.000 0 1574715499 0 non-token data: newest atime
0.000 0 0 0 non-token data: last journal sync atime
0.000 0 1574712064 0 non-token data: last expiry atime
0.000 0 812640 0 non-token data: last expire atime delta
0.000 0 204342 0 non-token data: last expire reduction count
Do note that every user has it's own DB ... can be overriden with
bayes_sql_override_username bayes
So if either "spamd" user or your own runs it ... you should get the same result ...
( I'm actually running it in PgSQL, can't remember what the file based is called )
mvh
Mikael Syska
---- On Sun, 24 Nov 2019 19:35:16 +0100 Jerry Malcolm <te...@malcolms.com> wrote ----
Thanks again to everyone who helped me get everything up and running
over the past couple of days.
Now that I have SA finding my bayes database, I'm curious about bayes
reporting. My bayes db is not new. I migrated my previous bayes db
from my old installation. And I've been training it with ~100+ new spam
messages in the past day. But I'm concerned that I'm not seeing
anything related to bayes in any scoring report. Nothing on any spam
message or good message over the past hundreds of message that have
arrived since getting bayes up and running in SA. I would expect at
least some messages to have a bayes rule fire, either positively or
negatively, or even just a 0.0 score based on bayes analysis. Is it
possible that after all of the setup I've done that bayes is still not
activated? Is there a way to force a bayes entry in the scoring summary
for an email just to confirm that it is indeed part of the scoring
process for spamd? I added "use_bayes 1" to the config just in case.
But no difference. I've got allow_tell activated, and training with an
email is giving me a good rc (It was correctly giving an error before I
added allow_tell). Just nothing in the scoring results for any email.
Re: Bayes
Posted by RW <rw...@googlemail.com>.
On Sun, 24 Nov 2019 12:35:16 -0600
Jerry Malcolm wrote:
> Thanks again to everyone who helped me get everything up and running
> over the past couple of days.
>
> Now that I have SA finding my bayes database, I'm curious about bayes
> reporting. My bayes db is not new. I migrated my previous bayes db
> from my old installation. And I've been training it with ~100+ new
> spam messages in the past day. But I'm concerned that I'm not seeing
> anything related to bayes in any scoring report.
Provided you have trained (by default) 200 ham and 200 spam, you should
get a BAYES_* rule on each scan.