You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by hg user <me...@gmail.com> on 2019/05/27 10:26:29 UTC
Is Bayes forgetting ?
I'm trying to understand what it is happening in my spamassassin 3.4.1
bayes system.
I make it learn a new message as spam and it learns it correctly, so that
the message is reported BAYES_50.
Unfortunately, after a few minutes, and with no new messages learnt
(autolearn=no), the message is reported as BAYES_00 !!!
I'm testing with command line spamassassin -t file.
Since I don't teach any new message to the bayes system, how can it change
idea on a message in a few minutes?
Please note that these messages are in italian.
I list the result of --dump magic since I noticed a very large difference
between nspam and nham... sincerely I can't interpret if these numbers are
good or not ...
Thank you
Francesco
$ sa-learn --dump magic
netset: cannot include 127.0.0.0/8 as it has already been included
netset: cannot include 0:0:0:0:0:0:0:1/128 as it has already been included
0.000 0 3 0 non-token data: bayes db version
0.000 0 1042011 0 non-token data: nspam
0.000 0 66472 0 non-token data: nham
0.000 0 663479 0 non-token data: ntokens
0.000 0 1558905855 0 non-token data: oldest atime
0.000 0 1558951293 0 non-token data: newest atime
0.000 0 0 0 non-token data: last journal sync
atime
0.000 0 1558948806 0 non-token data: last expiry atime
0.000 0 43200 0 non-token data: last expire atime
delta
0.000 0 191486 0 non-token data: last expire
reduction count
Re: Is Bayes forgetting ?
Posted by hg user <me...@gmail.com>.
From spamassassin manual:
*It's also worth noting that training with a very small quantity of ham,
will produce atrocious results. You should aim to train with at least the
same amount (or more if possible!) of ham data than spam.*
Actually it seems my spamassassin was trained with a lot of spam messages
(90% not in italian...)
Can this be a problem?
On Mon, May 27, 2019 at 12:26 PM hg user <me...@gmail.com> wrote:
>
> $ sa-learn --dump magic
> netset: cannot include 127.0.0.0/8 as it has already been included
> netset: cannot include 0:0:0:0:0:0:0:1/128 as it has already been included
> 0.000 0 3 0 non-token data: bayes db version
> 0.000 0 1042011 0 non-token data: nspam
> 0.000 0 66472 0 non-token data: nham
> 0.000 0 663479 0 non-token data: ntokens
> 0.000 0 1558905855 0 non-token data: oldest atime
> 0.000 0 1558951293 0 non-token data: newest atime
> 0.000 0 0 0 non-token data: last journal sync
> atime
> 0.000 0 1558948806 0 non-token data: last expiry atime
> 0.000 0 43200 0 non-token data: last expire atime
> delta
> 0.000 0 191486 0 non-token data: last expire
> reduction count
>
>
Re: Is Bayes forgetting ?
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
On 28.05.19 12:50, Jim Dunphy wrote:
>Attempting to answer all your questions related to zimbra
>
>Zimbra doesn't do any training until it is kicked off by cron for the zimbra user... Training a message as spam from the MUA interface sends the email to a special spam account... Training messages as not spam goes to a special ham account. You can pull those emails or use your admin interface to see what the users have trained before actual training occurs as a consequence. Look at your zimbra crontab entry to see when zmtrainsa runs which pulls the emails and trains them. With everything in zimbra, make sure you are the zimbra user before running any SA commands. If you want to train vast amounts of email as spam or ham, you can use the zmtrainsa and point it to a directory of messages. zmtrainsa is just a wrapper to get your zimbra spamassassin environment correct so that would show you sa-learn and the correct options (ie. paths), etc. Given this is really zimbra specific, probably better to ask this on the Zimbra administrator forums if you have specific questions so the rest of the community can learn.
>
>Note: it doesn't require anything special to run spamassassin in debug mode to test rules... just make sure you do it as the zimbra user so your environment is established.
I'm afraid this is not true, at least not by default:
May 29 11:58:10.809 [962] dbg: generic: Perl 5.016003,
PREFIX=/opt/zimbra/common,
DEF_RULES_DIR=/opt/zimbra/data/spamassassin/rules,
LOCAL_RULES_DIR=/opt/zimbra/data/spamassassin/localrules,
LOCAL_STATE_DIR=/opt/zimbra/data/spamassassin/state
May 29 11:55:48.515 [31733] dbg: config: internal_networks not configured,
using trusted_networks configuration for internal_networks; if you really
want internal_networks to only contain the required 127/8 add
'internal_networks !0/0' to your configuration
the SA version coming with zimbra does not seem have environment nor
configuration set to read proper config files when run by user in debug mode
/opt/zimbra/data/spamassassin/localrules/salocal.cf does contain them but
it's not read by default when running spamassassin
>----- On May 27, 2019, at 5:13 AM, hg user <me...@gmail.com> wrote:
>
>> Ok, sorry for my bad answers...
>
>> I test spamassassin results using a command line:
>> spamassassin -t /path/file
>> sometimes I add -D to look at the debug logs.
>
>> spamassassin is actually used via amavis, in a zimbra setup.
>
>> To teach sa-learn and to test the results via command line I use zimbra user.
>
>> On Mon, May 27, 2019 at 1:18 PM Matus UHLAR - fantomas < [
>> mailto:uhlar@fantomas.sk | uhlar@fantomas.sk ] > wrote:
>
>>> On 27.05.19 12:51, hg user wrote:
>>> >the Linux user is the same.
>
>>> the same as what?
>
>>> >Bayes db is on Linux.
>
>>> seems I wasn't clear at my question:
>>> How do you use spamassassin? milter, amavis, procmail filter, postfix
>>> filter ... ?
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Emacs is a complicated operating system without good text editor.
Re: Is Bayes forgetting ?
Posted by Jim Dunphy <ja...@aesir.com>.
Attempting to answer all your questions related to zimbra
Zimbra doesn't do any training until it is kicked off by cron for the zimbra user... Training a message as spam from the MUA interface sends the email to a special spam account... Training messages as not spam goes to a special ham account. You can pull those emails or use your admin interface to see what the users have trained before actual training occurs as a consequence. Look at your zimbra crontab entry to see when zmtrainsa runs which pulls the emails and trains them. With everything in zimbra, make sure you are the zimbra user before running any SA commands. If you want to train vast amounts of email as spam or ham, you can use the zmtrainsa and point it to a directory of messages. zmtrainsa is just a wrapper to get your zimbra spamassassin environment correct so that would show you sa-learn and the correct options (ie. paths), etc. Given this is really zimbra specific, probably better to ask this on the Zimbra administrator forums if you have specific questions so the rest of the community can learn.
Note: it doesn't require anything special to run spamassassin in debug mode to test rules... just make sure you do it as the zimbra user so your environment is established.
HTH,
Jim
----- On May 27, 2019, at 5:13 AM, hg user <me...@gmail.com> wrote:
> Ok, sorry for my bad answers...
> I test spamassassin results using a command line:
> spamassassin -t /path/file
> sometimes I add -D to look at the debug logs.
> spamassassin is actually used via amavis, in a zimbra setup.
> To teach sa-learn and to test the results via command line I use zimbra user.
> On Mon, May 27, 2019 at 1:18 PM Matus UHLAR - fantomas < [
> mailto:uhlar@fantomas.sk | uhlar@fantomas.sk ] > wrote:
>> On 27.05.19 12:51, hg user wrote:
>> >the Linux user is the same.
>> the same as what?
>> >Bayes db is on Linux.
>> seems I wasn't clear at my question:
>> How do you use spamassassin? milter, amavis, procmail filter, postfix
>> filter ... ?
>> --
>> Matus UHLAR - fantomas, [ mailto:uhlar@fantomas.sk | uhlar@fantomas.sk ] ; [
>> http://www.fantomas.sk/ | http://www.fantomas.sk/ ]
>> Warning: I wish NOT to receive e-mail advertising to this address.
>> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
>> Eagles may soar, but weasels don't get sucked into jet engines.
Re: Is Bayes forgetting ?
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
On 27.05.19 15:15, hg user wrote:
>I think, well... I suppose to be doing everything according to zimbra
>requirements.
does zimbra document how to run spamassassin test?
>I'm trying to undersand if it is possible that the bayes plugin reports
>different results within a few minutes and with no changes to the bayes
>db...
>
>Here are the commands, so that you may tell me if some parameter is wrong...
>
>spamassassin -t corpus/spam/phish-tst
>returns BAYES_00
>/opt/zimbra/common/bin/sa-learn --dbpath=<correct_path> -L --no-sync
>--spam corpus/spam/phish-tst
>spamassassin -t corpus/spam/phish-tst
>returns BAYES_50
>*wait a few minutes, no new messages learnt*
>spamassassin -t corpus/spam/phish-tst
>returns BAYES_00
apparently zimbra does more than just this. Did you try to ask in zimbra
forums first?
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
If Barbie is so popular, why do you have to buy her friends?
Re: Is Bayes forgetting ?
Posted by hg user <me...@gmail.com>.
I think, well... I suppose to be doing everything according to zimbra
requirements.
I'm trying to undersand if it is possible that the bayes plugin reports
different results within a few minutes and with no changes to the bayes
db...
Here are the commands, so that you may tell me if some parameter is wrong...
spamassassin -t corpus/spam/phish-tst
returns BAYES_00
/opt/zimbra/common/bin/sa-learn --dbpath=<correct_path> -L --no-sync
--spam corpus/spam/phish-tst
spamassassin -t corpus/spam/phish-tst
returns BAYES_50
*wait a few minutes, no new messages learnt*
spamassassin -t corpus/spam/phish-tst
returns BAYES_00
>
Re: Is Bayes forgetting ?
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
On 27.05.19 14:13, hg user wrote:
>Ok, sorry for my bad answers...
>
>I test spamassassin results using a command line:
>spamassassin -t /path/file
>sometimes I add -D to look at the debug logs.
>
>spamassassin is actually used via amavis, in a zimbra setup.
zimbra apparently runs under zimbra user with home set to /opt/zimbra.
your SA database is apparently in /opt/zimbra/data/amavisd/.spamassassin
and you should use sa-learn with apropriate --dbpath:
sa-learn --dbpath /opt/zimbra/data/amavisd/.spamassassin --dump magic
no idea how to check with SA. maybe create /opt/zimbra/.spamassassin/
symlink to /opt/zimbra/data/amavisd/.spamassassin and running
"spamassassin -t" under zimbra user would help.
>To teach sa-learn and to test the results via command line I use zimbra
>user.
zimbra has its own accounts used for training spam and ham.
You apparently should use those.
>On Mon, May 27, 2019 at 1:18 PM Matus UHLAR - fantomas <uh...@fantomas.sk>
>wrote:
>
>> On 27.05.19 12:51, hg user wrote:
>> >the Linux user is the same.
>>
>> the same as what?
>>
>> >Bayes db is on Linux.
>>
>> seems I wasn't clear at my question:
>> How do you use spamassassin? milter, amavis, procmail filter, postfix
>> filter ... ?
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
I drive way too fast to worry about cholesterol.
Re: Is Bayes forgetting ?
Posted by hg user <me...@gmail.com>.
Ok, sorry for my bad answers...
I test spamassassin results using a command line:
spamassassin -t /path/file
sometimes I add -D to look at the debug logs.
spamassassin is actually used via amavis, in a zimbra setup.
To teach sa-learn and to test the results via command line I use zimbra
user.
On Mon, May 27, 2019 at 1:18 PM Matus UHLAR - fantomas <uh...@fantomas.sk>
wrote:
> On 27.05.19 12:51, hg user wrote:
> >the Linux user is the same.
>
> the same as what?
>
> >Bayes db is on Linux.
>
> seems I wasn't clear at my question:
> How do you use spamassassin? milter, amavis, procmail filter, postfix
> filter ... ?
> --
> Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
> Warning: I wish NOT to receive e-mail advertising to this address.
> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
> Eagles may soar, but weasels don't get sucked into jet engines.
>
Re: Is Bayes forgetting ?
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
On 27.05.19 12:51, hg user wrote:
>the Linux user is the same.
the same as what?
>Bayes db is on Linux.
seems I wasn't clear at my question:
How do you use spamassassin? milter, amavis, procmail filter, postfix
filter ... ?
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Eagles may soar, but weasels don't get sucked into jet engines.
Re: Is Bayes forgetting ?
Posted by hg user <me...@gmail.com>.
Hi,
the Linux user is the same.
Bayes db is on Linux.
Re: Is Bayes forgetting ?
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
On 27.05.19 12:26, hg user wrote:
>I'm trying to understand what it is happening in my spamassassin 3.4.1
>bayes system.
>
>I make it learn a new message as spam and it learns it correctly, so that
>the message is reported BAYES_50.
>Unfortunately, after a few minutes, and with no new messages learnt
>(autolearn=no), the message is reported as BAYES_00 !!!
>I'm testing with command line spamassassin -t file.
hos do you use bayes? THe most common reason for this is that people use
different user for checking (e.g. amavis uses user 'amavis') than for
training (the current user).
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Depression is merely anger without enthusiasm.