You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Quinn Comendant <qu...@strangecode.com> on 2014/10/20 14:26:45 UTC
yahoo rcvd bug?
I'm getting FORGED_YAHOO_RCVD false positives for messages with yahoo received headers that do not match the search pattern defined in check_for_forged_yahoo_received_headers(). I'm using SpamAssassin 3.3.2 with latest rules as per `sa-update` rule channels `sought.rules.yerp.org` and `updates.spamassassin.org`.
The spamassassin rule that is firing:
* 1.6 FORGED_YAHOO_RCVD 'From' yahoo.com does not match 'Received' headers
The received-by header in question:
Received: from unknown (HELO nm46-vm10.bullet.mail.bf1.yahoo.com) (216.109.114.203)
Full mail headers available at https://cloudup.com/cbmG8tJF71k
And finally here's the `check_for_forged_yahoo_received_headers` function that parses this, which doesn't contain the correct regex for this hostname:
sub check_for_forged_yahoo_received_headers {
my ($self, $pms) = @_;
my $from = $pms->get('From:addr');
if ($from !~ /\byahoo\.com$/i) { return 0; }
my $rcvd = $pms->get('Received');
if ($pms->get("Resent-From") ne '' && $pms->get("Resent-To") ne '') {
my $xrcvd = $pms->get("X-Received");
$rcvd = $xrcvd if $xrcvd ne '';
}
$rcvd =~ s/\s+/ /gs; # just spaces, simplify the regexp
# not sure about this
#if ($rcvd !~ /from \S*yahoo\.com/) { return 0; }
if ($self->gated_through_received_hdr_remover($pms)) { return 0; }
# bug 3740: ignore bounces from Yahoo!. only honoured if the
# correct rDNS shows up in the trusted relay list, or first untrusted relay
#
# bug 4528: [ ip=68.142.202.54 rdns=mta122.mail.mud.yahoo.com
# helo=mta122.mail.mud.yahoo.com by=eclectic.kluge.net ident=
# envfrom= intl=0 id=49F2EAF13B auth= ]
#
if ($pms->{relays_trusted_str} =~ / rdns=\S+\.yahoo\.com /
|| $pms->{relays_untrusted_str} =~ /^[^\]]+ rdns=\S+\.yahoo\.com /)
{ return 0; }
if ($rcvd =~ /by web\S+\.mail\S*\.yahoo\.com via HTTP/) { return 0; }
if ($rcvd =~ /by smtp\S+\.yahoo\.com with SMTP/) { return 0; }
my $IP_ADDRESS = IP_ADDRESS;
if ($rcvd =~
/from \[$IP_ADDRESS\] by \S+\.(?:groups|scd|dcn)\.yahoo\.com with NNFMP/) {
return 0;
}
# used in "forward this news item to a friend" links. There's no better
# received hdrs to match on, unfortunately. I'm not sure if the next test is
# still useful, as a result.
#
# search for msgid <20...@xent.com>, subject "Yahoo!
# News Story - Top Stories", date Sep 29 2002 on
# <http://xent.com/pipermail/fork/> for an example.
#
if ($rcvd =~ /\bmailer\d+\.bulk\.scd\.yahoo\.com\b/
&& $from =~ /\@reply\.yahoo\.com$/i) { return 0; }
if ($rcvd =~ /by \w+\.\w+\.yahoo\.com \(\d+\.\d+\.\d+\/\d+\.\d+\.\d+\)(?: with ESMTP)? id \w+/) {
# possibly sent from "mail this story to a friend"
return 0;
}
return 1;
}
Re: yahoo rcvd bug?
Posted by Axb <ax...@gmail.com>.
On 10/20/2014 02:26 PM, Quinn Comendant wrote:
> Full mail headers available athttps://cloudup.com/cbmG8tJF71k
Pls put on pastebin - cloudup timesout
Re: yahoo rcvd bug?
Posted by Quinn Comendant <qu...@strangecode.com>.
So apparently on my system—qmail as per qmailtoaster.com—it is by design not to include the rDNS hostname in the "Received: …" header. See my discussion on the QMT list: <http://www.mail-archive.com/qmailtoaster-list@qmailtoaster.com/msg38313.html>.
So that degrades SA performance huh?
At least it's configurable, I'm going to enable lookups and see what happens.
Quinn
Re: yahoo rcvd bug?
Posted by Dave Funk <db...@engineering.uiowa.edu>.
On Mon, 20 Oct 2014, Quinn Comendant wrote:
> I'm getting FORGED_YAHOO_RCVD false positives for messages with yahoo received headers that do not match the search pattern defined in check_for_forged_yahoo_received_headers(). I'm using SpamAssassin 3.3.2 with latest rules as per `sa-update` rule channels `sought.rules.yerp.org` and `updates.spamassassin.org`.
>
> The spamassassin rule that is firing:
>
> * 1.6 FORGED_YAHOO_RCVD 'From' yahoo.com does not match 'Received' headers
>
> The received-by header in question:
>
> Received: from unknown (HELO nm46-vm10.bullet.mail.bf1.yahoo.com) (216.109.114.203)
>
> Full mail headers available at https://cloudup.com/cbmG8tJF71k
>
> And finally here's the `check_for_forged_yahoo_received_headers` function that parses this, which doesn't contain the correct regex for this hostname:
[snip..]
> return 1;
> }
You have two different rules that have fired there (FORGED_YAHOO_RCVD &
RDNS_NONE) because your MTA was not able to resolve that IP address to
its registered domain name.
The SA code correctly parsed the info that your MTA gave it, it's just
that info was incorrect either due to local DNS issues or a network issue.
Then because you (or somebody configuring your SA) has lowered the spam
threshold from 5.0 to 3.0 it caused a FP on this message.
I don't think that it is valid to delcare a bug in SA because of an issue
local to your system. (problematic MTA/DNS & local config choices).
I see that you also have a hit on URIBL_BLOCKED which tends to indicate
that you have local DNS issues that should be addressed.
suggestions:
1) work on improving your DNS system
2) put the spam threshold back to default to reduce FPs triggered by DNS
issues.
3) create a meta rule that takes the DKIM_VALID detection to nullify the
effect of that FORGED_YAHOO_RCVD (in case you cannot get your DNS to work
correctly).
If you lowered that spam threshold because of too many FNs, I think that
getting the DNS fixed so RBL tests work will take care of that too.
There have been plenty of posts to this list about URIBL_BLOCKED and how
to fix it.
--
Dave Funk University of Iowa
<dbfunk (at) engineering.uiowa.edu> College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{
Re: yahoo rcvd bug?
Posted by Reindl Harald <h....@thelounge.net>.
Am 20.10.2014 um 16:43 schrieb Axb:
> On 10/20/2014 04:37 PM, Reindl Harald wrote:
>>
>> Am 20.10.2014 um 16:25 schrieb Axb:
>>> On 10/20/2014 02:56 PM, Reindl Harald wrote:
>>>>> seems your recursor couldn't resolve it, so your MTA added unknown...
>>>>>
>>>>> Received: from nm46-vm10.bullet.mail.bf1.yahoo.com
>>>>> (nm46-vm10.bullet.mail.bf1.yahoo.com [216.109.114.203])
>>>>
>>>> what happens in case of a temporary DNS error?
>>>>
>>>> postfix anserwers in such cases with a 450
>>>> even with "unknown_hostname_reject_code = 550"
>>>
>>> only if you do tell Postfix to do so..
>>
>> no true
>
> oh yes it is...
>
>> http://www.postfix.org/postconf.5.html#reject_unknown_client_hostname
>>
>> The unknown_client_reject_code parameter specifies the response code for
>> rejected requests (default: 450). The reply is *always 450* in case the
>> address->name or name->address lookup failed due to a *temporary problem*
>
> reject_unknown_client_hostname
> reject_unknown_reverse_client_hostname
>
> are NOT default.
> had he set such check it would have not hit SA..
i asked *what does SA* in case of a *temporary* error and brought
postfix as *example* that it could be handeled different to a response
>>> not relevant..
>
>> it's relevant in case the OP had a temporary DNS error because no
>> nameserver responded resulting in a FP
>
> it resulted in a FORGED_YAHOO_RCVD hit - takes way more than that to
> trigger a FP
> anyway.. EOT...
> (why am I wasting my time?)
sorry for asking a simple question and bringing an example - you could
have just answered "not handling different and i don't" bother, frankly
that is the most hostile list i remember
Re: yahoo rcvd bug?
Posted by Axb <ax...@gmail.com>.
On 10/20/2014 04:37 PM, Reindl Harald wrote:
>
>
> Am 20.10.2014 um 16:25 schrieb Axb:
>> On 10/20/2014 02:56 PM, Reindl Harald wrote:
>>>> seems your recursor couldn't resolve it, so your MTA added unknown...
>>>>
>>>> works for me:
>>>>
>>>> Received: from nm46-vm10.bullet.mail.bf1.yahoo.com
>>>> (nm46-vm10.bullet.mail.bf1.yahoo.com [216.109.114.203])
>>>
>>> what happens in case of a temporary DNS error?
>>>
>>> postfix anserwers in such cases with a 450
>>> even with "unknown_hostname_reject_code = 550"
>>
>> only if you do tell Postfix to do so..
>
> no true
oh yes it is...
> http://www.postfix.org/postconf.5.html#reject_unknown_client_hostname
>
> The unknown_client_reject_code parameter specifies the response code for
> rejected requests (default: 450). The reply is *always 450* in case the
> address->name or name->address lookup failed due to a *temporary problem*
reject_unknown_client_hostname
reject_unknown_reverse_client_hostname
are NOT default.
had he set such check it would have not hit SA..
>> not relevant..
> it's relevant in case the OP had a temporary DNS error because no
> nameserver responded resulting in a FP
it resulted in a FORGED_YAHOO_RCVD hit - takes way more than that to
trigger a FP
anyway.. EOT...
(why am I wasting my time?)
Re: yahoo rcvd bug?
Posted by Reindl Harald <h....@thelounge.net>.
Am 20.10.2014 um 16:25 schrieb Axb:
> On 10/20/2014 02:56 PM, Reindl Harald wrote:
>>> seems your recursor couldn't resolve it, so your MTA added unknown...
>>>
>>> works for me:
>>>
>>> Received: from nm46-vm10.bullet.mail.bf1.yahoo.com
>>> (nm46-vm10.bullet.mail.bf1.yahoo.com [216.109.114.203])
>>
>> what happens in case of a temporary DNS error?
>>
>> postfix anserwers in such cases with a 450
>> even with "unknown_hostname_reject_code = 550"
>
> only if you do tell Postfix to do so..
no true
http://www.postfix.org/postconf.5.html#reject_unknown_client_hostname
The unknown_client_reject_code parameter specifies the response code for
rejected requests (default: 450). The reply is *always 450* in case the
address->name or name->address lookup failed due to a *temporary problem*
> not relevant..
it's relevant in case the OP had a temporary DNS error because no
nameserver responded resulting in a FP
Re: yahoo rcvd bug?
Posted by Axb <ax...@gmail.com>.
On 10/20/2014 02:56 PM, Reindl Harald wrote:
>
> Am 20.10.2014 um 14:51 schrieb Axb:
>> On 10/20/2014 02:26 PM, Quinn Comendant wrote:
>>> I'm getting FORGED_YAHOO_RCVD false positives for messages with yahoo
>>> received headers that do not match the search pattern defined in
>>> check_for_forged_yahoo_received_headers(). I'm using SpamAssassin
>>> 3.3.2 with latest rules as per `sa-update` rule channels
>>> `sought.rules.yerp.org` and `updates.spamassassin.org`.
>>>
>>> The spamassassin rule that is firing:
>>>
>>> * 1.6 FORGED_YAHOO_RCVD 'From' yahoo.com does not match
>>> 'Received' headers
>>>
>>> The received-by header in question:
>>>
>>> Received: from unknown (HELO nm46-vm10.bullet.mail.bf1.yahoo.com)
>>> (216.109.114.203)
>>
>> seems your recursor couldn't resolve it, so your MTA added unknown...
>>
>> works for me:
>>
>> Received: from nm46-vm10.bullet.mail.bf1.yahoo.com
>> (nm46-vm10.bullet.mail.bf1.yahoo.com [216.109.114.203])
>
> what happens in case of a temporary DNS error?
>
> postfix anserwers in such cases with a 450
> even with "unknown_hostname_reject_code = 550"
only if you do tell Postfix to do so.. not relevant..
Re: yahoo rcvd bug?
Posted by Reindl Harald <h....@thelounge.net>.
Am 20.10.2014 um 14:51 schrieb Axb:
> On 10/20/2014 02:26 PM, Quinn Comendant wrote:
>> I'm getting FORGED_YAHOO_RCVD false positives for messages with yahoo
>> received headers that do not match the search pattern defined in
>> check_for_forged_yahoo_received_headers(). I'm using SpamAssassin
>> 3.3.2 with latest rules as per `sa-update` rule channels
>> `sought.rules.yerp.org` and `updates.spamassassin.org`.
>>
>> The spamassassin rule that is firing:
>>
>> * 1.6 FORGED_YAHOO_RCVD 'From' yahoo.com does not match
>> 'Received' headers
>>
>> The received-by header in question:
>>
>> Received: from unknown (HELO nm46-vm10.bullet.mail.bf1.yahoo.com)
>> (216.109.114.203)
>
> seems your recursor couldn't resolve it, so your MTA added unknown...
>
> works for me:
>
> Received: from nm46-vm10.bullet.mail.bf1.yahoo.com
> (nm46-vm10.bullet.mail.bf1.yahoo.com [216.109.114.203])
what happens in case of a temporary DNS error?
postfix anserwers in such cases with a 450
even with "unknown_hostname_reject_code = 550"
Re: yahoo rcvd bug?
Posted by Axb <ax...@gmail.com>.
On 10/20/2014 02:26 PM, Quinn Comendant wrote:
> I'm getting FORGED_YAHOO_RCVD false positives for messages with yahoo received headers that do not match the search pattern defined in check_for_forged_yahoo_received_headers(). I'm using SpamAssassin 3.3.2 with latest rules as per `sa-update` rule channels `sought.rules.yerp.org` and `updates.spamassassin.org`.
>
> The spamassassin rule that is firing:
>
> * 1.6 FORGED_YAHOO_RCVD 'From' yahoo.com does not match 'Received' headers
>
> The received-by header in question:
>
> Received: from unknown (HELO nm46-vm10.bullet.mail.bf1.yahoo.com) (216.109.114.203)
>
seems your recursor couldn't resolve it, so your MTA added unknown...
works for me:
Received: from nm46-vm10.bullet.mail.bf1.yahoo.com
(nm46-vm10.bullet.mail.bf1.yahoo.com [216.109.114.203])