You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Merlin <ng...@fastmail.fm> on 2007/12/20 11:52:44 UTC
False positives with Bayes_99
Hi there,
I am running a well trusted travel community page that sends system
e-mails like register, notice on comments etc. to its opt-in signed up
users.
Since two days all E-Mails from that server get an aditional spam score
of 3.5!! by Bayes_99. I looked it up and found that Spamassasin believes
that it is to 99% spam by training from users. I believe there is more
to it, as I can not believe that
users mark such msges as spam. I also received another e-mail from
another community page that was marked with Bayes_99 despite that it
never has before. How come?! I looked into several red lists for my
server, but the server is not listed anywhere. The only thing I found is
that the server was not set with "reverse mapping" to the correct
domain, but to the one the hostmaster has set before (it is a root
server). Changed it yesterday to the domain name but still no change
today. Still wrong host. Does this have something to do with Bayes_99?
I am wondering how to get rid of this Bayes_99 thing and how to get to
Bayes_00 that would be more suitable for that e-mail. Do I have to
configure Postfix as the sending instance somehow with anything like
truested server lists, or with anything else I might have overlooked by
configuring it?
Here is a header of a false positive:
Subject: {SPAM 03.5} Feedback: lost password - please help
X-Spam: spam
X-Spam-score: 3.5
X-Spam-hits: BAYES_99 3.5, BAYES_USED global
X-Spam-source: IP='87.106.60.58',
Host='s15229619.onlinehome-server.info', Country='DE',
FromHeader='net', MailFrom='net'
Thank you for any help,
Merlin
--
Merlin
ngroups@fastmail.fm
--
http://www.fastmail.fm - One of many happy users:
http://www.fastmail.fm/docs/quotes.html
Re: False positives with Bayes_99
Posted by Merlin <ng...@fastmail.fm>.
On Thu, 20 Dec 2007 15:18:45 +0100, "Matthias Haegele"
<mh...@linuxrocks.dyndns.org> said:
> Merlin schrieb:
> > On Thu, 20 Dec 2007 03:08:34 -0800, "Merlin" <ng...@fastmail.fm> said:
> >>
> >>
> >> On Thu, 20 Dec 2007 11:59:32 +0100, "Matthias Haegele"
> >> <mh...@linuxrocks.dyndns.org> said:
> >>> Merlin schrieb:
> >>>> Hi there,
> >>>>
> >>>> I am running a well trusted travel community page that sends system
> >>>> e-mails like register, notice on comments etc. to its opt-in signed up
> >>>> users.
> >>>>
> >>>> Since two days all E-Mails from that server get an aditional spam score
> >>>> of 3.5!! by Bayes_99. I looked it up and found that Spamassasin believes
> >>>> that it is to 99% spam by training from users. I believe there is more
> >>>> to it, as I can not believe that
> >>>> users mark such msges as spam. I also received another e-mail from
> >>>> another community page that was marked with Bayes_99 despite that it
> >>>> never has before. How come?! I looked into several red lists for my
> >>>> server, but the server is not listed anywhere. The only thing I found is
> >>>> that the server was not set with "reverse mapping" to the correct
> >>>> domain, but to the one the hostmaster has set before (it is a root
> >>>> server). Changed it yesterday to the domain name but still no change
> >>>> today. Still wrong host. Does this have something to do with Bayes_99?
> >>>>
> >>>> I am wondering how to get rid of this Bayes_99 thing and how to get to
> >>>> Bayes_00 that would be more suitable for that e-mail. Do I have to
> >>>> configure Postfix as the sending instance somehow with anything like
> >>>> truested server lists, or with anything else I might have overlooked by
> >>>> configuring it?
> >>>>
> >>>> Here is a header of a false positive:
> >>>>
> >>>> Subject: {SPAM 03.5} Feedback: lost password - please help
> >>>> X-Spam: spam
> >>>> X-Spam-score: 3.5
> >>>> X-Spam-hits: BAYES_99 3.5, BAYES_USED global
> >>>> X-Spam-source: IP='87.106.60.58',
> >>>> Host='s15229619.onlinehome-server.info', Country='DE',
> >>>> FromHeader='net', MailFrom='net'
> >>>>
> >>>> Thank you for any help,
> >>> afaik the bayes results comes only from manual training and autolearn?
> >>> So the reverse dns, missing Pointer record is hit by another rule ...
> >>>
> >>> Perhaps you need to retrain the messages as ham (sa-learn --ham ...).
> >>> Or if your bayes-database is completely "poisoned" start from scratch.
> >>>
> >>> Perhaps you could show the bayes_mumble ...
> >>>
> >>>> Merlin
> >>>
> >>> --
> >>> Greetings & hth
> >>> MH
> >>>
> >>>
> >>> Dont send mail to: ubecatcher@linuxrocks.dyndns.org
> >>> --
> >>>
> >>
> >> Hi,
> >>
> >> thank you for your reply. I am not the one who can train ist. I am just
> >> running the server with
> >> the community that sends the messages. It is a big problem for me as if
> >> those e-mails do get false
> >> positive no more registration might be pssible etc.
> >>
> >> The funny thing is, that e-mails with almost identical content (for
> >> example notifications on forum
> >> replies) from other sites get even a Bayes_00 while mine get Bayes_99
> >> (that is true for the fastmail.fm e-mail
> >> provider). How come? Do you believe it has to do with the content, or
> >> the header? It must be the header as
> >> for example feedback msgs. that I receive through an online form also
> >> get marked with Bayes_99.
> >> The e-mails are sent through the PHPmailer class (opensource). I also
> >> looked there, but could not find a misconfig or so.
>
> Hmm. If you couldnt influence the training process and therefore cant
> rely on it,
> you probably dont want to use Bayes scores or at least lower BAYES_99?
>
> Perhaps you would like to use a pastebin-service like
> http://pastebin.com/
> and show us some "False Positive Samples" (feel free to exchange
> confidential parts, understandable plz).
>
> >> Thank you for any help,
> >>
> >> Merlin
>
>
> --
> Gruesse/Greetings
> MH
>
>
> Dont send mail to: ubecatcher@linuxrocks.dyndns.org
> --
>
Thank you for your reply. I have uploaded an example of the complete
e-mail that
got a...
Bayes_99: http://pastebin.com/db1f0425
Bayes_80: http://pastebin.com/da5a6714
This occures only since 2 days now. Most of the other mails I do get
inside my e-mail account is
with bayes_00 that even got a -2.x score. As those e-mails are extremly
important for my community
I would like to make sure that the members receive it. No idea why they
do not get a Bayes_00 as well.
Perhaps I have misconfigured the SMPT Server/ Postfix or PHPmailer or
the Linux server itself?
To make sure there is no misunderstanding, I am not running the server
that is classifying the e-mail
with Bayes_99, but the server that has sent that e-mail.
Best regards,
Merlin
--
Merlin
ngroups@fastmail.fm
--
http://www.fastmail.fm - Or how I learned to stop worrying and
love email again
Re: False positives with Bayes_99
Posted by Matthias Haegele <mh...@linuxrocks.dyndns.org>.
Merlin schrieb:
> On Thu, 20 Dec 2007 03:08:34 -0800, "Merlin" <ng...@fastmail.fm> said:
>>
>>
>> On Thu, 20 Dec 2007 11:59:32 +0100, "Matthias Haegele"
>> <mh...@linuxrocks.dyndns.org> said:
>>> Merlin schrieb:
>>>> Hi there,
>>>>
>>>> I am running a well trusted travel community page that sends system
>>>> e-mails like register, notice on comments etc. to its opt-in signed up
>>>> users.
>>>>
>>>> Since two days all E-Mails from that server get an aditional spam score
>>>> of 3.5!! by Bayes_99. I looked it up and found that Spamassasin believes
>>>> that it is to 99% spam by training from users. I believe there is more
>>>> to it, as I can not believe that
>>>> users mark such msges as spam. I also received another e-mail from
>>>> another community page that was marked with Bayes_99 despite that it
>>>> never has before. How come?! I looked into several red lists for my
>>>> server, but the server is not listed anywhere. The only thing I found is
>>>> that the server was not set with "reverse mapping" to the correct
>>>> domain, but to the one the hostmaster has set before (it is a root
>>>> server). Changed it yesterday to the domain name but still no change
>>>> today. Still wrong host. Does this have something to do with Bayes_99?
>>>>
>>>> I am wondering how to get rid of this Bayes_99 thing and how to get to
>>>> Bayes_00 that would be more suitable for that e-mail. Do I have to
>>>> configure Postfix as the sending instance somehow with anything like
>>>> truested server lists, or with anything else I might have overlooked by
>>>> configuring it?
>>>>
>>>> Here is a header of a false positive:
>>>>
>>>> Subject: {SPAM 03.5} Feedback: lost password - please help
>>>> X-Spam: spam
>>>> X-Spam-score: 3.5
>>>> X-Spam-hits: BAYES_99 3.5, BAYES_USED global
>>>> X-Spam-source: IP='87.106.60.58',
>>>> Host='s15229619.onlinehome-server.info', Country='DE',
>>>> FromHeader='net', MailFrom='net'
>>>>
>>>> Thank you for any help,
>>> afaik the bayes results comes only from manual training and autolearn?
>>> So the reverse dns, missing Pointer record is hit by another rule ...
>>>
>>> Perhaps you need to retrain the messages as ham (sa-learn --ham ...).
>>> Or if your bayes-database is completely "poisoned" start from scratch.
>>>
>>> Perhaps you could show the bayes_mumble ...
>>>
>>>> Merlin
>>>
>>> --
>>> Greetings & hth
>>> MH
>>>
>>>
>>> Dont send mail to: ubecatcher@linuxrocks.dyndns.org
>>> --
>>>
>>
>> Hi,
>>
>> thank you for your reply. I am not the one who can train ist. I am just
>> running the server with
>> the community that sends the messages. It is a big problem for me as if
>> those e-mails do get false
>> positive no more registration might be pssible etc.
>>
>> The funny thing is, that e-mails with almost identical content (for
>> example notifications on forum
>> replies) from other sites get even a Bayes_00 while mine get Bayes_99
>> (that is true for the fastmail.fm e-mail
>> provider). How come? Do you believe it has to do with the content, or
>> the header? It must be the header as
>> for example feedback msgs. that I receive through an online form also
>> get marked with Bayes_99.
>> The e-mails are sent through the PHPmailer class (opensource). I also
>> looked there, but could not find a misconfig or so.
Hmm. If you couldnt influence the training process and therefore cant
rely on it,
you probably dont want to use Bayes scores or at least lower BAYES_99?
Perhaps you would like to use a pastebin-service like http://pastebin.com/
and show us some "False Positive Samples" (feel free to exchange
confidential parts, understandable plz).
>> Thank you for any help,
>>
>> Merlin
--
Gruesse/Greetings
MH
Dont send mail to: ubecatcher@linuxrocks.dyndns.org
--
Re: False positives with Bayes_99
Posted by Merlin <ng...@fastmail.fm>.
On Thu, 20 Dec 2007 03:08:34 -0800, "Merlin" <ng...@fastmail.fm> said:
>
>
>
> On Thu, 20 Dec 2007 11:59:32 +0100, "Matthias Haegele"
> <mh...@linuxrocks.dyndns.org> said:
> > Merlin schrieb:
> > > Hi there,
> > >
> > > I am running a well trusted travel community page that sends system
> > > e-mails like register, notice on comments etc. to its opt-in signed up
> > > users.
> > >
> > > Since two days all E-Mails from that server get an aditional spam score
> > > of 3.5!! by Bayes_99. I looked it up and found that Spamassasin believes
> > > that it is to 99% spam by training from users. I believe there is more
> > > to it, as I can not believe that
> > > users mark such msges as spam. I also received another e-mail from
> > > another community page that was marked with Bayes_99 despite that it
> > > never has before. How come?! I looked into several red lists for my
> > > server, but the server is not listed anywhere. The only thing I found is
> > > that the server was not set with "reverse mapping" to the correct
> > > domain, but to the one the hostmaster has set before (it is a root
> > > server). Changed it yesterday to the domain name but still no change
> > > today. Still wrong host. Does this have something to do with Bayes_99?
> > >
> > > I am wondering how to get rid of this Bayes_99 thing and how to get to
> > > Bayes_00 that would be more suitable for that e-mail. Do I have to
> > > configure Postfix as the sending instance somehow with anything like
> > > truested server lists, or with anything else I might have overlooked by
> > > configuring it?
> > >
> > > Here is a header of a false positive:
> > >
> > > Subject: {SPAM 03.5} Feedback: lost password - please help
> > > X-Spam: spam
> > > X-Spam-score: 3.5
> > > X-Spam-hits: BAYES_99 3.5, BAYES_USED global
> > > X-Spam-source: IP='87.106.60.58',
> > > Host='s15229619.onlinehome-server.info', Country='DE',
> > > FromHeader='net', MailFrom='net'
> > >
> > > Thank you for any help,
> >
> > afaik the bayes results comes only from manual training and autolearn?
> > So the reverse dns, missing Pointer record is hit by another rule ...
> >
> > Perhaps you need to retrain the messages as ham (sa-learn --ham ...).
> > Or if your bayes-database is completely "poisoned" start from scratch.
> >
> > Perhaps you could show the bayes_mumble ...
> >
> > > Merlin
> >
> >
> > --
> > Greetings & hth
> > MH
> >
> >
> > Dont send mail to: ubecatcher@linuxrocks.dyndns.org
> > --
> >
>
>
> Hi,
>
> thank you for your reply. I am not the one who can train ist. I am just
> running the server with
> the community that sends the messages. It is a big problem for me as if
> those e-mails do get false
> positive no more registration might be pssible etc.
>
> The funny thing is, that e-mails with almost identical content (for
> example notifications on forum
> replies) from other sites get even a Bayes_00 while mine get Bayes_99
> (that is true for the fastmail.fm e-mail
> provider). How come? Do you believe it has to do with the content, or
> the header? It must be the header as
> for example feedback msgs. that I receive through an online form also
> get marked with Bayes_99.
> The e-mails are sent through the PHPmailer class (opensource). I also
> looked there, but could not find a misconfig or so.
>
> Thank you for any help,
>
> Merlin
> --
> Merlin
> ngroups@fastmail.fm
>
> --
> http://www.fastmail.fm - A no graphics, no pop-ups email service
>
Hi,
thank you for your reply. I am not the one who can train ist. I am just
running the server with
the community that sends the messages. It is a big problem for me as if
those e-mails do get false
positive no more registration might be pssible etc.
The funny thing is, that e-mails with almost identical content (for
example notifications on forum
replies) from other sites get even a Bayes_00 while mine get Bayes_99
(that is true for the fastmail.fm e-mail
provider). How come? Do you believe it has to do with the content, or
the header? It must be the header as
for example feedback msgs. that I receive through an online form also
get marked with Bayes_99.
The e-mails are sent through the PHPmailer class (opensource). I also
looked there, but could not find a misconfig or so.
Thank you for any help,
Merlin
--
Merlin
ngroups@fastmail.fm
--
http://www.fastmail.fm - A no graphics, no pop-ups email service
Re: False positives with Bayes_99
Posted by Matthias Haegele <mh...@linuxrocks.dyndns.org>.
Merlin schrieb:
> Hi there,
>
> I am running a well trusted travel community page that sends system
> e-mails like register, notice on comments etc. to its opt-in signed up
> users.
>
> Since two days all E-Mails from that server get an aditional spam score
> of 3.5!! by Bayes_99. I looked it up and found that Spamassasin believes
> that it is to 99% spam by training from users. I believe there is more
> to it, as I can not believe that
> users mark such msges as spam. I also received another e-mail from
> another community page that was marked with Bayes_99 despite that it
> never has before. How come?! I looked into several red lists for my
> server, but the server is not listed anywhere. The only thing I found is
> that the server was not set with "reverse mapping" to the correct
> domain, but to the one the hostmaster has set before (it is a root
> server). Changed it yesterday to the domain name but still no change
> today. Still wrong host. Does this have something to do with Bayes_99?
>
> I am wondering how to get rid of this Bayes_99 thing and how to get to
> Bayes_00 that would be more suitable for that e-mail. Do I have to
> configure Postfix as the sending instance somehow with anything like
> truested server lists, or with anything else I might have overlooked by
> configuring it?
>
> Here is a header of a false positive:
>
> Subject: {SPAM 03.5} Feedback: lost password - please help
> X-Spam: spam
> X-Spam-score: 3.5
> X-Spam-hits: BAYES_99 3.5, BAYES_USED global
> X-Spam-source: IP='87.106.60.58',
> Host='s15229619.onlinehome-server.info', Country='DE',
> FromHeader='net', MailFrom='net'
>
> Thank you for any help,
afaik the bayes results comes only from manual training and autolearn?
So the reverse dns, missing Pointer record is hit by another rule ...
Perhaps you need to retrain the messages as ham (sa-learn --ham ...).
Or if your bayes-database is completely "poisoned" start from scratch.
Perhaps you could show the bayes_mumble ...
> Merlin
--
Greetings & hth
MH
Dont send mail to: ubecatcher@linuxrocks.dyndns.org
--
Re: False positives with Bayes_99
Posted by "John D. Hardin" <jh...@impsec.org>.
On Thu, 20 Dec 2007, Merlin wrote:
> I looked it up and found that Spamassasin believes that it is to
> 99% spam by training from users. I believe there is more to it, as
> I can not believe that users mark such msges as spam.
An unfortunate reality of system administration is that most people
are idiots.
As a practical note, this is why it is critical to keep the corpus
around if you're doing manual training - so that you can find and fix
mistrained messages, and retrain from scratch if you need to.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
"Bother," said Pooh as he struggled with /etc/sendmail.cf, "it never
does quite what I want. I wish Christopher Robin was here."
-- Peter da Silva in a.s.r
-----------------------------------------------------------------------
5 days until Christmas