You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by bu...@bugzilla.spamassassin.org on 2008/11/24 13:09:42 UTC

[Bug 6022] New: False positives on TVD_RCVD_IP and TVD_RCVD_IP4 rules on Yahoo mail

https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6022

           Summary: False positives on TVD_RCVD_IP and TVD_RCVD_IP4 rules on
                    Yahoo mail
           Product: Spamassassin
           Version: 3.2.5
          Platform: Other
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P5
         Component: Rules
        AssignedTo: dev@spamassassin.apache.org
        ReportedBy: francisrussell3001-sabugzilla@yahoo.co.uk


Created an attachment (id=4393)
 --> (https://issues.apache.org/SpamAssassin/attachment.cgi?id=4393)
Example Headers

I receive my email though BT which uses Yahoo to provide its email service. It
seems the Yahoo servers don't perform a reverse DNS lookup on the connection
and always puts an IP address in the header which triggers the TVD_RCVD_IP and
TVD_RCVD_IP4 rules on all messages.

I've attached headers from an email to show the problem. In the example it's
not an issue, but it biases the score by 5.1 which is significant for other
emails.  Obviously, I can disable these rules, or increase my required score,
but this is really a problem if I want to forward my mail somewhere else that
uses a Spamassassin installation that isn't under my control.


-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6022] False positives on TVD_RCVD_IP and TVD_RCVD_IP4 rules on Yahoo mail

Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6022


Jan Ingvoldstad <fr...@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |frettled@gmail.com




--- Comment #1 from Jan Ingvoldstad <fr...@gmail.com>  2009-01-07 06:13:10 PST ---
(Note: I'm not a SpamAssassin developer.)

I can confirm that this is an issue for quite a few people, and would of course
love to see a fix.

But I don't think that it's correct for a piece of software like SpamAssassin
to add specific exceptions for specific e-mail providers, that would probably
mean that SpamAssassin needs a central list of Yahoo's e-mail servers' IP
address ranges for all of Yahoo's e-mail operations.

This is therefore not so much a bug in SpamAssassin as it is a problem with
Yahoo's e-mail policies.

I would first complain to BT that "your e-mail services have a problem causing
legimitate messages to be tagged as spam all over the world", and try to get
other BT users to file similar reports. If enough customers complain that their
mail is tagged as spam, BT may either pressure Yahoo into changing their
policies, or BT may switch to another provider.

For ISPs, it's possible to turn on DKIM verification checking in SpamAssassin
and assign an appropriate negative score, but that will also increase the
amount of spam that gets through.


-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6022] False positives on TVD_RCVD_IP and TVD_RCVD_IP4 rules on Yahoo mail

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6022

Mark Martinec <Ma...@ijs.si> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P5                          |P3
   Target Milestone|Undefined                   |3.3.2

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6022] False positives on TVD_RCVD_IP and TVD_RCVD_IP4 rules on Yahoo mail

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6022

John Wilcock <jo...@tradoc.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |john@tradoc.fr

--- Comment #3 from John Wilcock <jo...@tradoc.fr> 2010-04-21 10:57:43 EDT ---
This problem is not limited to yahoo, nor to 3.2.5, but I don't have the privs
to change the title and version. 

For example, this Received: header corresponds to a message from an e-mail
client on my phone. It hit TVD_RCVD_IP and TVD_RCVD_IP4, yet also hit
ALL_TRUSTED (and rightly so, being authenticated on my server).

Received: from 10.156.53.80 (unknown [62.201.142.1])
    (using TLSv1 with cipher AES256-SHA (256/256 bits))
    (No client certificate requested)
    (Authenticated sender: xxxx@yyyyy.zzz)
    by www.yyyyy.zzz (Postfix) with ESMTPSA id B1DD233400A;
    Wed, 14 Apr 2010 09:56:09 +0200 (CEST)

Sure, the email client on my phone shouldn't really be EHLOing with a plain IP
address, but nevertheless this sort of rule shouldn't be hitting in parallel
with ALL_TRUSTED. 

I'd second the suggestion to convert these rules to use
X-Spam-Relays-Untrusted.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6022] False positives on TVD_RCVD_IP and TVD_RCVD_IP4 rules on Yahoo mail

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6022

--- Comment #5 from John Wilcock <jo...@tradoc.fr> 2010-04-22 08:42:07 EDT ---
(In reply to comment #4)
> I've also realised that the TVD_RCVD_IP rules have significant overlap with
> FH_HELO_EQ_D_D_D_D which already uses X-Spam-Relays-External. 
> 
> Perhaps the best course of action is actually to delete TVD_RCVD_IP and
> TVD_RCVD_IP4 ?

Damn, hit send too soon. 

To be more precise, TVD_RCVD_IP overlaps with FH_HELO_EQ_D_D_D_D and/or
HELO_DYNAMIC_IPADDR2 and/or HELO_DYNAMIC_SPLIT_IP on all actual spam I see
here.  All of these are high-scoring rules, and TVD_RCVD_IP should probably
therefore be deleted. 

I don't see any hits for TVD_RCVD_IP4 as I block non-authenticated naked IPs at
SMTP time, before SA gets a look at the message, but if the corpora show that
it is worth keeping it should no doubt be updated to use
X-Spam-Relays-External.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6022] False positives on TVD_RCVD_IP and TVD_RCVD_IP4 rules on Yahoo mail

Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6022

--- Comment #4 from John Wilcock <jo...@tradoc.fr> 2010-04-22 08:17:21 EDT ---
I've also realised that the TVD_RCVD_IP rules have significant overlap with
FH_HELO_EQ_D_D_D_D which already uses X-Spam-Relays-External. 

Perhaps the best course of action is actually to delete TVD_RCVD_IP and
TVD_RCVD_IP4 ?

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6022] False positives on TVD_RCVD_IP and TVD_RCVD_IP4 rules on Yahoo mail

Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6022

Henrik Krohns <he...@hege.li> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |hege@hege.li
         Resolution|                            |FIXED

--- Comment #6 from Henrik Krohns <he...@hege.li> 2011-05-01 19:40:21 UTC ---
Due to overlaps with bunch of similar rules, I'm going to settle this at
informative 0.001. It does seem to have some unique hits, but using Received
imo is bad for this. Let user decide.

Trunk:
Sending        rules/50_scores.cf
Transmitting file data .
Committed revision 1098394.

3.3:
Sending        rules/50_scores.cf
Transmitting file data .
Committed revision 1098395.

-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 6022] False positives on TVD_RCVD_IP and TVD_RCVD_IP4 rules on Yahoo mail

Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6022





--- Comment #2 from mouss <mo...@netoyen.net>  2009-01-10 03:21:56 PST ---
IMHO, it is a bug. the regexp does not deal with the various Received formats
that Recevived.pm already handles. I don't think it is reasonable
- to force ISPs to do reverse dns lookup. 
- to force MTA developpers to use "Received: from $helo ..." format


Since code to parse Received headers is already in Received.pm, I suggest using
X-Spam-Relays headers instead of a regexp. for example:

header RCVD_HELO_NAKED X-Spam-Relays-Untrusted =~ /helo=[\d\.]+ /

This may still catch ham if the server didn't include the HELO string in a way
that can be parsed by SA. but if such formats are discovered, they can be
handled in Received.pm. 


-- 
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.