You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Greg Troxel <gd...@ir.bbn.com> on 2010/05/04 21:44:33 UTC
odd FPs
I use spamassass-milter and reject at about 8 points. Normally this is
fine. I just got a few false positives. One message got 10.9 points
(from earthlink, from an earthlink user). DKIM_FORGED is my own rule
and wrong, now fixed, for 2 points, but that's 8.9 points still. I
don't understand why so many rules are firing. This has been happening
to me also from someone with a gmail address from google's servers.
BAYES_40,DKIM_FORGED,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DOS_OUTLOOK_TO_MX,HELO_NO_DOMAIN,RCVD_IN_PBL,RCVD_IN_SORBS_DUL,RDNS_NONE,UNPARSEABLE_RELAY
SpamAssassin version 3.3.1
running on Perl version 5.10.1
with:
gdt 7 ~ > l /var/spamassassin/3.003001/
total 14
drwxr-xr-x 2 root wheel 512 Apr 26 00:00 khop-bl_sa_khopesh_com
-rw-r--r-- 1 root wheel 115 Apr 26 00:00 khop-bl_sa_khopesh_com.cf
drwxr-xr-x 2 root wheel 512 May 4 00:00 sought_rules_yerp_org
-rw-r--r-- 1 root wheel 119 May 4 00:00 sought_rules_yerp_org.cf
drwxr-xr-x 2 root wheel 2048 Apr 17 00:00 updates_spamassassin_org
-rw-r--r-- 1 root wheel 2537 Apr 17 00:00 updates_spamassassin_org.cf
gdt 6 ~ > l /var/spamassassin/3.003001/updates_spamassassin_org
total 878
-rw-r--r-- 1 root wheel 8315 Apr 17 00:00 10_default_prefs.cf
[snip]
I put the gmail user in my whitelist and had her resend, and then it got
-100.9 points (-100 for wl, -0.9 on own merits) - which seems totally
fine. Am I suffering from transient DNS lossage? Due to
http://www.theregister.co.uk/2010/04/13/dnssec/
a day early? (Box is netbsd with modern bind - it's fine with dnssec.)
Any other clues?
Re: [sa] odd FPs
Posted by Greg Troxel <gd...@ir.bbn.com>.
Charles Gregory <cg...@hwcn.org> writes:
> On Tue, 4 May 2010, Greg Troxel wrote:
>> Thanks - I did pretty much understand the tests. What I'm boggled
>> about is that they suddenly started firing, and then now suddenly do
>> not.
>
> This is perfectly consistent with the explanation I offered at the
> beginning of this thread. A legitimate Google MX was temporarily
> blacklisted. Given that it was hitting dialup Lists, I would guess
> that maybe Google was (re)assigned an IP block that was previously
> dynamic.
The problem was that the synthetic Received: line spamass-milter should
have added was not added, and thus SA thought this was direct-to-MX from
the person's computer. Adding '_' to milter_connect_macros fixed things.
Re: [sa] odd FPs
Posted by Charles Gregory <cg...@hwcn.org>.
On Tue, 4 May 2010, Greg Troxel wrote:
> Thanks - I did pretty much understand the tests. What I'm boggled about
> is that they suddenly started firing, and then now suddenly do not.
This is perfectly consistent with the explanation I offered at the
beginning of this thread. A legitimate Google MX was temporarily
blacklisted. Given that it was hitting dialup Lists, I would guess that
maybe Google was (re)assigned an IP block that was previously dynamic.
- Charles
Re: [sa] odd FPs
Posted by Greg Troxel <gd...@ir.bbn.com>.
Karsten Bräckelmann <gu...@rudersport.de> writes:
> On Tue, 2010-05-04 at 17:37 -0400, Greg Troxel wrote:
>> Karsten Bräckelmann <gu...@rudersport.de> writes:
>
>> > Translations of the rules? An Outlook user submitted directly to your MX
>> > without using his own SMTP. The SMTP HELO looks bad. The mail has been
>> > submitted from an IP, that's not supposed to do this: It's listed in
>> > PBL, the Policy Block List and DUL, a Dial-Up List.
>
> Greg, hope my explanation was insightful, and you now understand what
> they mean -- and that they all are playing in the same ball-park.
Thanks - I did pretty much understand the tests. What I'm boggled about
is that they suddenly started firing, and then now suddenly do not.
>> I don't have the headers because the mail was rejected. The users that
>
> Yeah, figured as much. But there isn't much we can do without the
> headers...
Understood - it was more "is anyone else suddenly seeing this, or is it
me"?
> Maybe add a specific rule, and have the user send a test message
> triggering that one rule, to offset the score? Get headers that way. Or,
> well, don't reject.
I added the person to the whitelist and had her retry. That message
came in at -100.9, so it was -0.9 without the whitelist points. I was
hoping to see all the same tests.
>> sent the mail are just normal people who wouldn't know how to configure
>
> Hmm, wait. PBL and SORBS DUL must not be used with deep parsing. And so
> we don't. Any chance, there's a bug with your MTA and generated Received
> headers? Or extensive trusted networks going berserk?
Everything has been stable, but Peter Alfredsen (thanks!) just pointed
me to:
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6103
which I am reading. Do people know if the consequence is occasional
unparseable received lines with this bug - certainly mostly it works.
But the line being missing can make it appear as direct-to-MX mail.
Re: [sa] odd FPs
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Tue, 2010-05-04 at 17:37 -0400, Greg Troxel wrote:
> Karsten Bräckelmann <gu...@rudersport.de> writes:
> > Translations of the rules? An Outlook user submitted directly to your MX
> > without using his own SMTP. The SMTP HELO looks bad. The mail has been
> > submitted from an IP, that's not supposed to do this: It's listed in
> > PBL, the Policy Block List and DUL, a Dial-Up List.
Greg, hope my explanation was insightful, and you now understand what
they mean -- and that they all are playing in the same ball-park.
> I don't have the headers because the mail was rejected. The users that
Yeah, figured as much. But there isn't much we can do without the
headers...
Maybe add a specific rule, and have the user send a test message
triggering that one rule, to offset the score? Get headers that way. Or,
well, don't reject.
> sent the mail are just normal people who wouldn't know how to configure
Hmm, wait. PBL and SORBS DUL must not be used with deep parsing. And so
we don't. Any chance, there's a bug with your MTA and generated Received
headers? Or extensive trusted networks going berserk?
> direct-to-MX sending. I did have the person send me the bounced mail,
> and I can see that the DSN was generated by googlemail.com
>
> headers in the bounce included:
>
> Received: by 10.231.171.83 with SMTP id g19mr1762180ibz.61.1272980385360;
> Tue, 04 May 2010 06:39:45 -0700 (PDT)
> Return-Path: <ja...@gmail.com>
> Received: from janePC (pool-x.y.z.w.bstnma.fios.verizon.net [redacted fios ip address that looks ok])
> by mx.google.com with ESMTPS id ct37sm1283572ibb.19.2010.05.04.06.39.42
> (version=TLSv1/SSLv3 cipher=RC4-MD5);
> Tue, 04 May 2010 06:39:43 -0700 (PDT)
>
> and it was received from
>
> mail-iw0-f173.google.com[209.85.223.173]
>
> so it seems that it really was google trying to send it to me.
>
> Both the google address and the fios address seem relatively clean (now)
> on blacklists.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: [sa] odd FPs
Posted by Greg Troxel <gd...@ir.bbn.com>.
Karsten Bräckelmann <gu...@rudersport.de> writes:
> On Tue, 2010-05-04 at 15:48 -0400, Charles Gregory wrote:
>> On Tue, 4 May 2010, Greg Troxel wrote:
>> > I use spamassass-milter and reject at about 8 points. Normally this is
>> > fine. I just got a few false positives.
>
>> > DOS_OUTLOOK_TO_MX,HELO_NO_DOMAIN,RCVD_IN_PBL,RCVD_IN_SORBS_DUL,RDNS_NONE
>
> It appears the user sent the mail *directly* to your MX from his dial-up
> IP, which he sure is not supposed to. But to confirm this, we would need
> the message's full headers.
>
> Translations of the rules? An Outlook user submitted directly to your MX
> without using his own SMTP. The SMTP HELO looks bad. The mail has been
> submitted from an IP, that's not supposed to do this: It's listed in
> PBL, the Policy Block List and DUL, a Dial-Up List.
>
>> > UNPARSEABLE_RELAY
>
> Yup, need headers. Either the above is correct, or some broken Received
> header triggered all the rules.
I don't have the headers because the mail was rejected. The users that
sent the mail are just normal people who wouldn't know how to configure
direct-to-MX sending. I did have the person send me the bounced mail,
and I can see that the DSN was generated by googlemail.com
headers in the bounce included:
Received: by 10.231.171.83 with SMTP id g19mr1762180ibz.61.1272980385360;
Tue, 04 May 2010 06:39:45 -0700 (PDT)
Return-Path: <ja...@gmail.com>
Received: from janePC (pool-x.y.z.w.bstnma.fios.verizon.net [redacted fios ip address that looks ok])
by mx.google.com with ESMTPS id ct37sm1283572ibb.19.2010.05.04.06.39.42
(version=TLSv1/SSLv3 cipher=RC4-MD5);
Tue, 04 May 2010 06:39:43 -0700 (PDT)
and it was received from
mail-iw0-f173.google.com[209.85.223.173]
so it seems that it really was google trying to send it to me.
Both the google address and the fios address seem relatively clean (now)
on blacklists.
Re: [sa] odd FPs
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Tue, 2010-05-04 at 15:48 -0400, Charles Gregory wrote:
> On Tue, 4 May 2010, Greg Troxel wrote:
> > I use spamassass-milter and reject at about 8 points. Normally this is
> > fine. I just got a few false positives.
> > DOS_OUTLOOK_TO_MX,HELO_NO_DOMAIN,RCVD_IN_PBL,RCVD_IN_SORBS_DUL,RDNS_NONE
It appears the user sent the mail *directly* to your MX from his dial-up
IP, which he sure is not supposed to. But to confirm this, we would need
the message's full headers.
Translations of the rules? An Outlook user submitted directly to your MX
without using his own SMTP. The SMTP HELO looks bad. The mail has been
submitted from an IP, that's not supposed to do this: It's listed in
PBL, the Policy Block List and DUL, a Dial-Up List.
> > UNPARSEABLE_RELAY
Yup, need headers. Either the above is correct, or some broken Received
header triggered all the rules.
> Your list of 'matched' rules includes several DNS blacklists (PBL, SORBS).
> Occasionally a gmail or earthlink server gets abused, and is temporarily
> blacklisted. IF they dn't fix it themselves, you may have to write to the
> blacklist maintainers and request removal of the IP....
You'll have a really hard time getting dynamic, dial-up IP pool space
off of PBL or SORBS DUL... ;) Even more so, if you are not the owner
ISP.
Nope, de-listing is not the solution. The listings themselves are not
the problem.
guenther
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: [sa] odd FPs
Posted by Charles Gregory <cg...@hwcn.org>.
On Tue, 4 May 2010, Greg Troxel wrote:
> I use spamassass-milter and reject at about 8 points. Normally this is
> fine. I just got a few false positives.
> BAYES_40,DKIM_FORGED,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DOS_OUTLOOK_TO_MX,HELO_NO_DOMAIN,RCVD_IN_PBL,RCVD_IN_SORBS_DUL,RDNS_NONE,UNPARSEABLE_RELAY
Your list of 'matched' rules includes several DNS blacklists (PBL, SORBS).
Occasionally a gmail or earthlink server gets abused, and is temporarily
blacklisted. IF they dn't fix it themselves, you may have to write to the
blacklist maintainers and request removal of the IP....
- C
Re: odd FPs
Posted by Greg Troxel <gd...@ir.bbn.com>.
Peter Alfredsen <pe...@gmail.com> writes:
> [I don't remember if this address is subscribed, so CCing Greg directly]
> On Tue, 04 May 2010 15:44:33 -0400
> Greg Troxel <gd...@ir.bbn.com> wrote:
>
>>
>> I use spamassass-milter and reject at about 8 points.
> [...]
>> UNPARSEABLE_RELAY
>
> That test has a habit of firing with spamass-milter if spamass-milter is
> not properly set up (missing milter macros) or if the 'fix
> auto-generated received header' patch isn't applied.
>
> See https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6103 for
> more info.
>
> It would also explain all the DNSBLs firing if the unparseable
> received-header was skipped and tests commenced on the Received: header
> added by the sending MTA. In your later mails you didn't give an IP,
> but, to take an example pool-98-110-190-205.bstnma.fios.verizon.net is
> listed in the blocklists that hit this FP.
I am using postfix, and should have said that.
I found that '_' was not in my postfix milter config as a default (this
seems wrong), and adding it per the link you sent fixes the problem.
Looking at my old logs, I now think I've been gettting UNPARSEABLE_RELAY
most of the time, and only losing when mail from a dynamic address
hitting many blocklists comes in. Otherwise, the failure to parse the
milter-generated line doesn't push the score too high.
So this in main.cf helped a lot:
# Add _ for spamass-milter per http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=510665
# Add i because spamass-milter complains, but to no avail.
milter_connect_macros = i j {daemon_name} v _
# {if_name} (recommended, not in source)