You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Andreas Ntaflos <da...@pseudoterminal.org> on 2008/02/22 15:35:52 UTC
AWL scores high after receiving spam "from myself"?
Hello list,
this is my first post here, although I have been happily using spamassassin
for years now.
I noticed something unsettling some time ago, and yesterday I think I found
the cause.
What I noticed was that when sending mail from one of my addresses to a
mailing list (or to myself) it would, upon retrieving it (using getmail and
spamc), often get a very high AWL score. Here is an example from a posting to
the freebsd-test mailing list:
X-Spam-Checker-Version: SpamAssassin 3.2.4 (2008-01-01) on pseudoterminal.org
X-Spam-Level: ****
X-Spam-Status: No, score=4.9 required=5.0
tests=[AWL=11.504,BAYES_00=-2.599,RCVD_IN_DNSWL_MED=-4,SPF_PASS=-0.001]
<dns:53.83.147.69.list.dnswl.org> [127.0.9.2]
<dns:freebsd.org?type=MX> [10 mx1.freebsd.org.]
<dns:freebsd.org> [69.147.83.40] autolearn=ham version=3.2.4
...
From: Andreas Ntaflos <my...@my_mail_provider.org>
On AWL it scored over 11 points and only by means of the various other tests
the message barely got under the spam threshold.
Naturally I was a little worried. Then, after reading up on AWL once again, I
got the idea to look through my spam folder and check whether I got any
spam "from" myself, i.e. where the From: header field indicated that the spam
was sent from my_address@my_mail_provider.org.
Of course I found one, because it doesn't seem uncommon for spammers to make
spam seem to come from the recipient:
X-Spam-Flag: YES
X-Spam-Checker-Version: SpamAssassin 3.2.4 (2008-01-01) on pseudoterminal.org
X-Spam-Level: ****************
X-Spam-Status: Yes, score=16.4 required=5.0
tests=[AWL=0.363,BAYES_99=3.5,HTML_IMAGE_ONLY_32=1.778,HTML_MESSAGE=...]
...
From: <my...@my_mail_provider.org>
To: <my...@my_mail_provider.org>
To spamassassin this spam appears to come from myself. It scored a low AWL but
over 16 points all in all so the next message received from
my_address@my_mail_provider.org would certainly get high AWL score.
My questions are these: did I get this right? Is that really what seems to be
happening? If so, how do I handle such a scenario? When it is so easy to
forge header fields does it even make sense to have an AWL that assigns
scores based on where the mail *appears* to be coming from?
Or am I looking in the completely wrong direction here?
Any help appreciated!
Thanks in advance,
Andreas
--
Andreas Ntaflos
Vienna, Austria
GPG Fingerprint: 6234 2E8E 5C81 C6CB E5EC 7E65 397C E2A8 090C A9B4
Re: AWL scores high after receiving spam "from myself"?
Posted by Andreas Ntaflos <da...@pseudoterminal.org>.
On Friday 22 February 2008 23:37:29 René Berber wrote:
> > Should I post the contents of both local.cf and user_prefs? They don't
> > contain anything special as far as I can see, but something definitely
> > feels wrong with my configuration. Why else would the AWL test get such
> > scores?
>
> AWL is probably not the culprit, as I said, it follows not leads.
Thank you for the reply and your time.
It has been about a week now since I removed the problematic address
(my_address@my_mail_provider.org) from the whitelist database and started
over. Initial tests have proved positive, no wrong AWL scores. The trust path
is correct by the way, no ALL_TRUSTED tests fire nor do I observe any of the
symptoms described on the wiki page.
But that was a week ago, and now I am back to the square one it seems. I just
posted to the Dovecot mailing list and found that when retrieving the message
from the remote mailserver (the one that hosts the problematic address) via
getmail the AWL test got a score of over 9.5.
Looking through my Received Spam folder I see lots of spams which seem to have
come from me, i.e. "From: my_address@my_mail_provider.org".
Now as far as I understand AWL looks at both the sender address
(my_address@my_mail_provider.org) and the IP the mail came from, right?
So it would seem that Spamassassin on my server looks at the sender address
(my_address@my_mail_provider.org) and the IP address of the server the
(possible) spam comes from. In my case the only IP address that could be
looked at is the IP address of the remote mailserver, i.e. that of
my_mail_provider.org (85.214.xx.yy). This is clearly not the desired
behaviour.
That would explain why the AWL score would become ever higher with every spam
(that has my address in the From: field) received on the remote mailserver
and then retrieved by me on my local mailserver. The mail address/IP address
pair would always be the same, no matter where the original spam originated
from.
I hope I could make clear what I am thinking. Am I thinking correctly? Is this
what is happening? If so, how do I solve this problem?
I really can't be having all legitimate mail sent to mailing lists by me end
up in the Spam folder just because some spammers put my address in the From:
field.
I'd really appreciate any further insight on this.
Andreas
--
Andreas "daff" Ntaflos
Vienna, Austria
GPG Fingerprint: 6234 2E8E 5C81 C6CB E5EC 7E65 397C E2A8 090C A9B4
Re: AWL scores high after receiving spam "from myself"?
Posted by René Berber <r....@computer.org>.
Andreas Ntaflos wrote:
[snip]
>> When you remove the email address you should not see an AWL score, if
>> you still see it then you didn't remove it. Remember that the AWL could
>> be global or per user (even if the "user" is the one running amavisd in
>> your case), dependending on your spamassassin configuration.
>
> Right, that's what I thought, too. I removed the address, sent a mail message
> to freebsd-test@freebsd.org, and AWL got no score. Half an hour later I sent
> another message and AWL got a negative score of about -1.1. Two hours later I
> sent yet another message and now AWL was at +0.370. And a message I sent just
> now got a score of +0.277. Is that normal?
Looks normal. How the AWL score moves depends on the total score given
to all the emails from that address, it's just a moving average that
includes the current message's score.
So, how did it get positive? You have to check the log, record total
scores and see what is hitting, the AWL follows, not leads.
My guess is that you are using something that scores high, like RBL
checks, Botnet, both in combination (sometimes they are redundant).
[snip]
> On that note: checking the AWL scores for mail messages to *this* list from
> that address (daff@pseudoterminal.org) shows that they are also a bit
> high-ish (the first message got +2.45, the second +2.25).
Same situation, you have to see what spamassassin rules are hitting.
> I am not sure if that is wrong, but I am getting more and more confused about
> the AWL. I never had problems with it before but recently it really seems to
> score badly only for messages sent by me!
Perhaps your domain or IP address got added to a blacklist. Only way to
know is checking the rules that hit.
> Should I post the contents of both local.cf and user_prefs? They don't contain
> anything special as far as I can see, but something definitely feels wrong
> with my configuration. Why else would the AWL test get such scores?
AWL is probably not the culprit, as I said, it follows not leads.
--
René Berber
Re: AWL scores high after receiving spam "from myself"?
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Fri, 2008-02-22 at 23:12 +0100, Andreas Ntaflos wrote:
> Right, that's what I thought, too. I removed the address, sent a mail message
> to freebsd-test@freebsd.org, and AWL got no score.
To be expected, first mail since removing from AWL, no previous records.
> Half an hour later I sent another message and AWL got a negative score
> of about -1.1.
The second message got a higher score (without AWL) and AWL joined in,
since it learned previously about a lower score.
> Two hours later I sent yet another message and now AWL was at +0.370.
This message scored slightly below your average of the previous two
posts.
> And a message I sent just now got a score of +0.277. Is that normal?
Same. And yes, this looks normal. Details depend on the actual score,
though, but given the numbers above abs(AWL) is constantly decreasing,
which is to be expected with an average over all messages, where the
scores are about similar.
AWL is a score *averaging* technique, based on previous messages. It is
not static, nor designed to only assign negative values.
http://wiki.apache.org/spamassassin/AutoWhitelist
guenther
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: AWL scores high after receiving spam "from myself"?
Posted by Andreas Ntaflos <da...@pseudoterminal.org>.
Thank you too for you reply!
On Friday 22 February 2008 22:26:07 René Berber wrote:
> Andreas Ntaflos wrote:
>
> [snip]
>
> > So I added my mailserver to the trusted_networks but after removing that
> > particularly troublesome address from the whitelist
> > (spamassassin --remove-addr-from-whitelist) and a few tests it seems that
> > AWL again scores in the wrong direction. Should I also add the remote
> > mailserver that is final destination for that troublesome address to
> > trusted_networks?
>
> No, don't add the remote mailserver.
Ok.
> When you remove the email address you should not see an AWL score, if
> you still see it then you didn't remove it. Remember that the AWL could
> be global or per user (even if the "user" is the one running amavisd in
> your case), dependending on your spamassassin configuration.
Right, that's what I thought, too. I removed the address, sent a mail message
to freebsd-test@freebsd.org, and AWL got no score. Half an hour later I sent
another message and AWL got a negative score of about -1.1. Two hours later I
sent yet another message and now AWL was at +0.370. And a message I sent just
now got a score of +0.277. Is that normal?
Amavisd is not involved in checking incoming mail that is retrieved via
getmail. It passes the message directly to spamc. Amavisd only checks
incoming mail for the domain the mailserver is final destination for, which
is for the domain of the address I use to post to this list
(pseudoterminal.org).
On that note: checking the AWL scores for mail messages to *this* list from
that address (daff@pseudoterminal.org) shows that they are also a bit
high-ish (the first message got +2.45, the second +2.25).
I am not sure if that is wrong, but I am getting more and more confused about
the AWL. I never had problems with it before but recently it really seems to
score badly only for messages sent by me!
Should I post the contents of both local.cf and user_prefs? They don't contain
anything special as far as I can see, but something definitely feels wrong
with my configuration. Why else would the AWL test get such scores?
Thanks again for your reply and your time!
Andreas
--
Andreas Ntaflos
Vienna, Austria
GPG Fingerprint: 6234 2E8E 5C81 C6CB E5EC 7E65 397C E2A8 090C A9B4
Re: AWL scores high after receiving spam "from myself"?
Posted by René Berber <r....@computer.org>.
Andreas Ntaflos wrote:
[snip]
> So I added my mailserver to the trusted_networks but after removing that
> particularly troublesome address from the whitelist
> (spamassassin --remove-addr-from-whitelist) and a few tests it seems that AWL
> again scores in the wrong direction. Should I also add the remote mailserver
> that is final destination for that troublesome address to trusted_networks?
No, don't add the remote mailserver.
When you remove the email address you should not see an AWL score, if
you still see it then you didn't remove it. Remember that the AWL could
be global or per user (even if the "user" is the one running amavisd in
your case), dependending on your spamassassin configuration.
--
René Berber
Re: AWL scores high after receiving spam "from myself"?
Posted by Andreas Ntaflos <da...@pseudoterminal.org>.
On Friday 22 February 2008 17:52:13 Rosenbaum, Larry M. wrote:
> > From: Andreas Ntaflos [mailto:daff@pseudoterminal.org]
> > To spamassassin this spam appears to come from myself. It scored a low
> > AWL but
> > over 16 points all in all so the next message received from
> > my_address@my_mail_provider.org would certainly get high AWL score.
> >
> > My questions are these: did I get this right? Is that really what seems
> > to be
> > happening? If so, how do I handle such a scenario? When it is so easy
> > to
> > forge header fields does it even make sense to have an AWL that assigns
> > scores based on where the mail *appears* to be coming from?
>
> The AWL classifies its history by both return address and IP. It sounds
> like in your case it is using the wrong IP, which may indicate problems
> with your trust path. Please see
>
> http://wiki.apache.org/spamassassin/TrustPath
Thank you for your reply! Unfortunately I'm not sure how setting
trusted_networks will help me, or how to test if it does.
As far as I understand none of the symptoms described in the wikipage you
linked are observed in my problematic scenario? Are there any other reasons
why AWL would continuously score in the wrong direction (i.e. positive)?
Maybe I should explain my setup a little further. I, as many others nowadays,
have several email addresses and use a single mailserver (under my control)
to retrieve mails (with getmail, and getmail putting retrieved mail through
external filters such as spamc and clamscan) from several other mailservers
(not under my control). I use my mailserver to send out mails for these
addresses, using postfix (with SASL auth) and amavisd (amavisd is configured
to bypass spam and virus checks for users who have authenticated successfully
through SASL).
So I added my mailserver to the trusted_networks but after removing that
particularly troublesome address from the whitelist
(spamassassin --remove-addr-from-whitelist) and a few tests it seems that AWL
again scores in the wrong direction. Should I also add the remote mailserver
that is final destination for that troublesome address to trusted_networks?
What else can I check to solve that problem? What else can I read to
understand the problem better? Because now I am not sure anymore that I *do*
really understand. Please forgive my ignorance.
Andreas
--
Andreas Ntaflos
Vienna, Austria
GPG Fingerprint: 6234 2E8E 5C81 C6CB E5EC 7E65 397C E2A8 090C A9B4
RE: AWL scores high after receiving spam "from myself"?
Posted by "Rosenbaum, Larry M." <ro...@ornl.gov>.
> From: Andreas Ntaflos [mailto:daff@pseudoterminal.org]
> To spamassassin this spam appears to come from myself. It scored a low
> AWL but
> over 16 points all in all so the next message received from
> my_address@my_mail_provider.org would certainly get high AWL score.
>
> My questions are these: did I get this right? Is that really what seems
> to be
> happening? If so, how do I handle such a scenario? When it is so easy
> to
> forge header fields does it even make sense to have an AWL that assigns
> scores based on where the mail *appears* to be coming from?
The AWL classifies its history by both return address and IP. It sounds like in your case it is using the wrong IP, which may indicate problems with your trust path. Please see
http://wiki.apache.org/spamassassin/TrustPath