Workaround for problems with dynamic IP + mail.gmx.net + fetchmail

[Jörg Mensmann <jo...@bitplanet.de>]

Hi all,

just yesterday I stumbled across the problem with dynamic IPs and
mail.gmx.net, as discussed a few days ago ("Simple mail from Dynamic IP
listed as spam").

The situation is as follows:
  dynamic ip -> mail.gmx.net (auth'd) -> pop.gmx.net -> fetchmail -> SA

As a minimal test-case, I sent a mail from my local machine (dynamic IP)
via smarthost mail.gmx.net and then retrieved it using fetchmail:

--------schnipp--------
Received: from localhost ([127.0.0.1] helo=mydomain.local)
	by mydomain.local with esmtp (Exim 4.63)
	(envelope-from <my...@gmx.de>)
	id 1H1rj6-0006sn-7E
	for me@mydomain.local; Tue, 02 Jan 2007 23:03:20 +0100
Delivered-To: GMX delivery to my.name@gmx.de
Received: from pop.gmx.net [213.165.64.22]
	by mydomain.local with POP3 (fetchmail-6.3.4)
	for <me...@mydomain.local> (single-drop); Tue, 02 Jan 2007 23:03:20 +0100 (CET)
Received: (qmail invoked by alias); 02 Jan 2007 22:02:47 -0000
Received: from pD9FFEA32.dip.t-dialin.net (EHLO mydomain.local) [217.255.234.50]
  by mail.gmx.net (mp054) with SMTP; 02 Jan 2007 23:02:47 +0100
X-Authenticated: #123456
Received: from me by mydomain.local with local (Exim 4.63)
	(envelope-from <my...@gmx.de>)
	id 1H1riX-0006sS-Un
	for my.name@gmx.de; Tue, 02 Jan 2007 23:02:45 +0100
To: my.name@gmx.de
Subject: Test
From: "Me" <my...@gmx.de>

Test.
--------schnapp--------

Now "spamassassin -D -t" (version 3.1.7 on Fedora Core 6) thinks that
the mail was directly sent from a dynamic IP, without using a relay,
because the Received-header does not mention any authentication (which
*is* happening):

--------schnipp--------
dbg: received-header: found fetchmail marker, restarting parse
dbg: dns: looking up PTR record for '217.255.234.50'
dbg: dns: PTR for '217.255.234.50': 'pD9FFEA32.dip.t-dialin.net'
dbg: received-header: parsed as [ 
     ip=217.255.234.50 rdns=pD9FFEA32.dip.t-dialin.net
     helo=mydomain.local by=mail.gmx.net ident= envfrom= intl=0 id= auth= ]
dbg: dns: looking up A records for 'mail.gmx.net'
dbg: dns: A records for 'mail.gmx.net': 213.165.64.20 213.165.64.21
dbg: received-header: 'by' mail.gmx.net has public IP 213.165.64.20
dbg: received-header: 'by' mail.gmx.net has public IP 213.165.64.21
dbg: received-header: relay 217.255.234.50 trusted? no internal? no
dbg: metadata: X-Spam-Relays-Trusted:
dbg: metadata: X-Spam-Relays-Untrusted: [ 
     ip=217.255.234.50 rdns=pD9FFEA32.dip.t-dialin.net
     helo=mydomain.local by=mail.gmx.net ident= envfrom= intl=0 id= auth= ]
dbg: metadata: X-Spam-Relays-Internal:
dbg: metadata: X-Spam-Relays-External: [ 
     ip=217.255.234.50 rdns=pD9FFEA32.dip.t-dialin.net
     helo=mydomain.local by=mail.gmx.net ident= envfrom= intl=0 id= auth= ]

...

 2.0 RCVD_IN_SORBS_DUL      RBL: SORBS: sent directly from dynamic IP address
                            [217.255.234.50 listed in dnsbl.sorbs.net]
 1.7 RCVD_IN_NJABL_DUL      RBL: NJABL: dialup sender did non-local SMTP
                            [217.255.234.50 listed in combined.njabl.org]
--------schnapp--------

As a simple workaround, I hard-coded a test into Received.pm:

--------schnipp--------
--- Mail/SpamAssassin/Message/Metadata/Received.pm
+++ Mail/SpamAssassin/Message/Metadata/Received.pm.jm
@@ -200,6 +200,13 @@
       if ($relay->{auth}) {
 	dbg("received-header: authentication method ".$relay->{auth});
 	$inferred_as_trusted = 1;
+      } else {
+        # workaround for server doing authentication but not setting
+        # received-header accordingly
+        if ($relay->{by} =~ /^mail\.gmx\.(de|net)$/) {
+          dbg("received-header: authentication by trusted server ".$relay->{by});
+          $inferred_as_trusted = 1;
+        }
       }
 
       # can we use DNS?  If not, we cannot use this algorithm, as we
--------schnapp--------

With this, ALL_TRUSTED is triggered and all seems fine.

Daryl's msa_networks-patch seemed to be the perfect solution, with
setting mail.gmx.net as a trusted MSA which is known to authenticate all
users. Unfortunately, msa_networks does not make any difference here,
because mail.gmx.net is never tested. Only the dynamic IP is compared
with the MSA list. With the following patch, also the "received by" is
checked:

--------schnipp--------
--- Mail/SpamAssassin/Message/Metadata/Received.pm
+++ Mail/SpamAssassin/Message/Metadata/Received.pm.jm
@@ -266,6 +266,11 @@
 	    dbg("received-header: 'by' ".$relay->{by}." has private IP $ip");
 	    $found_rsvd = 1;
 	  }
+	  
+	  if ($msa->contains_ip($ip)) {
+	    dbg("received-header: 'by' ".$relay->{by}." is in msa_networks");
+	    $inferred_as_trusted = 1;
+	  }
 	}
 
 	if ($found_rsvd && !$found_non_rsvd) {
--------schnapp--------

ciao
  Jörg