You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Ben Whyall <be...@whyall-systems.co.uk> on 2015/09/09 16:03:42 UTC
Problem with Bayes
Hi
I have a mail setup serving multiple domains that is running on ubuntu
lts 14.04.03 lts.
Spamassassin 3.4.0
Perl 5.18.2
Exim 4.82
We have bayes stored in a mysql database
When spamassassin is run from exim, it is always returning a hit on rule
Bayes_00 with a score of 0.0000, however if you run spamassassin <
message, it will return a different bayes score.
in the /etc/defaults/spamassassin file we have the following
OPTIONS="-p 987 -s /var/log/spamd.log --nouser-config --helper-home-dir
-H -u spamd"
the output of sa-learn -u spamd --dump magic is
0.000 0 3 0 non-token data: bayes db version
0.000 0 13794 0 non-token data: nspam
0.000 0 5619 0 non-token data: nham
0.000 0 1157620 0 non-token data: ntokens
0.000 0 1398925984 0 non-token data: oldest atime
0.000 0 1441801878 0 non-token data: newest atime
0.000 0 0 0 non-token data: last journal
sync atime
0.000 0 1441797057 0 non-token data: last expiry atime
0.000 0 5529600 0 non-token data: last expire
atime delta
0.000 0 783449 0 non-token data: last expire
reduction count
Could anyone suggest how I might find out what is going awry ?
Ben
--
Sent from Postbox
<https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campaign=reach>
--
Sent from Postbox
<https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campaign=reach>
Re: Problem with Bayes
Posted by Reindl Harald <h....@thelounge.net>.
Am 09.09.2015 um 16:03 schrieb Ben Whyall:
> I have a mail setup serving multiple domains that is running on ubuntu
> lts 14.04.03 lts.
>
> Spamassassin 3.4.0
> Perl 5.18.2
> Exim 4.82
>
> We have bayes stored in a mysql database
>
> When spamassassin is run from exim, it is always returning a hit on rule
> Bayes_00 with a score of 0.0000, however if you run spamassassin <
> message, it will return a different bayes score.
sound like exim is running spamassassin as a different user and like
using a different bayes
however, i observed that BAYES_xx for whatever reason *heavily* depends
on Received-headers and so results are often differetn between spamc and
running as milter for example
another example: in case there are no reveived headers at all bayes will
never ever hit 999, put a purely random, valid received header on top
and the same sample hits 999
observed that by stripa ton of headers out of our training-corpus with
"formail" and rebuild bayes, hence now there is on single static
Received-heaer on top of all samples and so they pass our masstests as
expected
Re: Problem with Bayes
Posted by Ben Whyall <be...@whyall-systems.co.uk>.
Hi
Here is an explanation of how exim is talking to Spamassassin in my
setup
http://www.exim.org/exim-html-current/doc/html/spec_html/ch-content_scanning_at_acl_time.html
I suspect that as we have spam = nobody, SA was running the queries as
nobody and as such not correctly using the right spam corpus.
Which the override statement is sorting.
Ben
> Matus UHLAR - fantomas <ma...@fantomas.sk>
> 09 September 2015 15:55via Postbox
> <https://www.postbox-inc.com/?utm_source=email&utm_medium=sumlink&utm_campaign=reach>
>>> Ben Whyall <ma...@whyall-systems.co.uk>
>>> 09 September 2015 15:03via Postbox
>>> <https://www.postbox-inc.com/?utm_source=email&utm_medium=sumlink&utm_campaign=reach>
>>>
>
>>> When spamassassin is run from exim, it is always returning a hit on
>>> rule Bayes_00 with a score of 0.0000, however if you run
>>> spamassassin < message, it will return a different bayes score.
>
> that loooks like spamassassin called from exim uses different
> username(s)...
>
>>> Axb <ma...@gmail.com>
>>> 09 September 2015 15:10via Postbox
>>> <https://www.postbox-inc.com/?utm_source=email&utm_medium=sumlink&utm_campaign=reach>
>>>
>>>
>>> No idea how Exim interfaces to SA but...
>
> ... so this is the most important question.
>
> On 09.09.15 15:21, Ben Whyall wrote:
>> I didnt have that option but I did have bayes_sql_username set
>>
>> I have added the sql_override as well.
>
> I don't think you should set such options without understsanding what
> it may
> cause.
> it may work, but also may fsck up spam filtering.
>
>> I have restarted and will see what happens.
>>
>> I am using the ubuntu exim-daemon heavy and it calls spamd directly.
>
> that means how?
>
> Ben Whyall <ma...@whyall-systems.co.uk>
> 09 September 2015 15:21via Postbox
> <https://www.postbox-inc.com/?utm_source=email&utm_medium=sumlink&utm_campaign=reach>
> Hi
>
> I didnt have that option but I did have bayes_sql_username set
>
> I have added the sql_override as well.
>
> I have restarted and will see what happens.
>
> I am using the ubuntu exim-daemon heavy and it calls spamd directly.
>
> Ben
>
>
> Ben Whyall <ma...@whyall-systems.co.uk>
> 09 September 2015 15:03via Postbox
> <https://www.postbox-inc.com/?utm_source=email&utm_medium=sumlink&utm_campaign=reach>
> Hi
>
> I have a mail setup serving multiple domains that is running on ubuntu
> lts 14.04.03 lts.
>
> Spamassassin 3.4.0
> Perl 5.18.2
> Exim 4.82
>
> We have bayes stored in a mysql database
>
> When spamassassin is run from exim, it is always returning a hit on
> rule Bayes_00 with a score of 0.0000, however if you run spamassassin
> < message, it will return a different bayes score.
>
> in the /etc/defaults/spamassassin file we have the following
>
> OPTIONS="-p 987 -s /var/log/spamd.log --nouser-config
> --helper-home-dir -H -u spamd"
>
> the output of sa-learn -u spamd --dump magic is
>
> 0.000 0 3 0 non-token data: bayes db version
> 0.000 0 13794 0 non-token data: nspam
> 0.000 0 5619 0 non-token data: nham
> 0.000 0 1157620 0 non-token data: ntokens
> 0.000 0 1398925984 0 non-token data: oldest atime
> 0.000 0 1441801878 0 non-token data: newest atime
> 0.000 0 0 0 non-token data: last journal
> sync atime
> 0.000 0 1441797057 0 non-token data: last expiry atime
> 0.000 0 5529600 0 non-token data: last expire
> atime delta
> 0.000 0 783449 0 non-token data: last expire
> reduction count
>
> Could anyone suggest how I might find out what is going awry ?
>
> Ben
>
--
Sent from Postbox
<https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campaign=reach>
Re: Problem with Bayes
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
>On 09/09/2015 04:55 PM, Matus UHLAR - fantomas wrote:
>>On 09.09.15 15:21, Ben Whyall wrote:
>>>I didnt have that option but I did have bayes_sql_username set
>>>
>>>I have added the sql_override as well.
>>
>>I don't think you should set such options without understsanding what it
>>may
>>cause.
>>it may work, but also may fsck up spam filtering.
On 09.09.15 17:05, Axb wrote:
>Did I take it for granted that the op would check the docs
>(http://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Conf.txt)
>before adding the option?
I would first search for answer to the question, how does calling SA from
exim work...
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
2B|!2B, that's a question!
Re: Problem with Bayes
Posted by Axb <ax...@gmail.com>.
On 09/09/2015 04:55 PM, Matus UHLAR - fantomas wrote:
> On 09.09.15 15:21, Ben Whyall wrote:
>> I didnt have that option but I did have bayes_sql_username set
>>
>> I have added the sql_override as well.
>
> I don't think you should set such options without understsanding what it
> may
> cause.
> it may work, but also may fsck up spam filtering.
Did I take it for granted that the op would check the docs
(http://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Conf.txt)
before adding the option?
Re: Problem with Bayes
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
>>Ben Whyall <ma...@whyall-systems.co.uk>
>>09 September 2015 15:03via Postbox <https://www.postbox-inc.com/?utm_source=email&utm_medium=sumlink&utm_campaign=reach>
>>When spamassassin is run from exim, it is always returning a hit on
>>rule Bayes_00 with a score of 0.0000, however if you run
>>spamassassin < message, it will return a different bayes score.
that loooks like spamassassin called from exim uses different username(s)...
>>Axb <ma...@gmail.com>
>>09 September 2015 15:10via Postbox <https://www.postbox-inc.com/?utm_source=email&utm_medium=sumlink&utm_campaign=reach>
>>
>>No idea how Exim interfaces to SA but...
... so this is the most important question.
On 09.09.15 15:21, Ben Whyall wrote:
>I didnt have that option but I did have bayes_sql_username set
>
>I have added the sql_override as well.
I don't think you should set such options without understsanding what it may
cause.
it may work, but also may fsck up spam filtering.
>I have restarted and will see what happens.
>
>I am using the ubuntu exim-daemon heavy and it calls spamd directly.
that means how?
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Silvester Stallone: Father of the RISC concept.
Re: Problem with Bayes
Posted by Ben Whyall <be...@whyall-systems.co.uk>.
Hi
I didnt have that option but I did have bayes_sql_username set
I have added the sql_override as well.
I have restarted and will see what happens.
I am using the ubuntu exim-daemon heavy and it calls spamd directly.
Ben
> Axb <ma...@gmail.com>
> 09 September 2015 15:10via Postbox
> <https://www.postbox-inc.com/?utm_source=email&utm_medium=sumlink&utm_campaign=reach>
>
>
> No idea how Exim interfaces to SA but...
>
> Do you have
> bayes_sql_override_username
> in local.cf ?
> if no, try adding that and restart spamd
>
> h2h
> Ben Whyall <ma...@whyall-systems.co.uk>
> 09 September 2015 15:03via Postbox
> <https://www.postbox-inc.com/?utm_source=email&utm_medium=sumlink&utm_campaign=reach>
> Hi
>
> I have a mail setup serving multiple domains that is running on ubuntu
> lts 14.04.03 lts.
>
> Spamassassin 3.4.0
> Perl 5.18.2
> Exim 4.82
>
> We have bayes stored in a mysql database
>
> When spamassassin is run from exim, it is always returning a hit on
> rule Bayes_00 with a score of 0.0000, however if you run spamassassin
> < message, it will return a different bayes score.
>
> in the /etc/defaults/spamassassin file we have the following
>
> OPTIONS="-p 987 -s /var/log/spamd.log --nouser-config
> --helper-home-dir -H -u spamd"
>
> the output of sa-learn -u spamd --dump magic is
>
> 0.000 0 3 0 non-token data: bayes db version
> 0.000 0 13794 0 non-token data: nspam
> 0.000 0 5619 0 non-token data: nham
> 0.000 0 1157620 0 non-token data: ntokens
> 0.000 0 1398925984 0 non-token data: oldest atime
> 0.000 0 1441801878 0 non-token data: newest atime
> 0.000 0 0 0 non-token data: last journal
> sync atime
> 0.000 0 1441797057 0 non-token data: last expiry atime
> 0.000 0 5529600 0 non-token data: last expire
> atime delta
> 0.000 0 783449 0 non-token data: last expire
> reduction count
>
> Could anyone suggest how I might find out what is going awry ?
>
> Ben
>
--
Sent from Postbox
<https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campaign=reach>
Re: Problem with Bayes
Posted by Axb <ax...@gmail.com>.
On 09/09/2015 04:03 PM, Ben Whyall wrote:
> Hi
>
> I have a mail setup serving multiple domains that is running on ubuntu
> lts 14.04.03 lts.
>
> Spamassassin 3.4.0
> Perl 5.18.2
> Exim 4.82
>
> We have bayes stored in a mysql database
>
> When spamassassin is run from exim, it is always returning a hit on rule
> Bayes_00 with a score of 0.0000, however if you run spamassassin <
> message, it will return a different bayes score.
>
> in the /etc/defaults/spamassassin file we have the following
>
> OPTIONS="-p 987 -s /var/log/spamd.log --nouser-config --helper-home-dir
> -H -u spamd"
>
> the output of sa-learn -u spamd --dump magic is
>
> 0.000 0 3 0 non-token data: bayes db version
> 0.000 0 13794 0 non-token data: nspam
> 0.000 0 5619 0 non-token data: nham
> 0.000 0 1157620 0 non-token data: ntokens
> 0.000 0 1398925984 0 non-token data: oldest atime
> 0.000 0 1441801878 0 non-token data: newest atime
> 0.000 0 0 0 non-token data: last journal
> sync atime
> 0.000 0 1441797057 0 non-token data: last expiry atime
> 0.000 0 5529600 0 non-token data: last expire
> atime delta
> 0.000 0 783449 0 non-token data: last expire
> reduction count
>
> Could anyone suggest how I might find out what is going awry ?
>
> Ben
>
No idea how Exim interfaces to SA but...
Do you have
bayes_sql_override_username
in local.cf ?
if no, try adding that and restart spamd
h2h