You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by bu...@bugzilla.spamassassin.org on 2009/09/11 20:55:32 UTC
[Bug 6196] New: FH_HELO_ENDS_DOT rule is overly aggressive in
assigning a spam score
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6196
Summary: FH_HELO_ENDS_DOT rule is overly aggressive in
assigning a spam score
Product: Spamassassin
Version: unspecified
Platform: Other
URL: http://forums.cpanel.net/f43/fh_helo_ends_dot-118261.h
tml
OS/Version: Linux
Status: NEW
Severity: normal
Priority: P3
Component: Rules
AssignedTo: dev@spamassassin.apache.org
ReportedBy: tpawlowski@fortressitx.com
A customer is having most of his emails that are sent out from his mail server
being marked as spam by the receiving server due the following:
X-Spam-Status: Yes, hits=5.903 tagged_above=1 required=5
tests=FH_HELO_ENDS_DOT=3.02, HTML_MESSAGE=0.001, MIME_HTML_ONLY=1.672,
SPF_NEUTRAL=1.21
X-Spam-Level: *****
One of the cPanel developers has weighed in on the issue in the aforementioned
URL and believes that the rule in question:
50_scores.cf:score FH_HELO_ENDS_DOT 3.599 3.020 1.395 2.308
scores matching headers far too heavily given that a FQDN technically is
supposed to end in a dot in the first place.
Has there been any discussion about this in the past? I searched SA Bugzilla
but did not come up with anything.
Thanks in advance for your attention to this matter.
Regards,
Tom Pawlowski
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6196] FH_HELO_ENDS_DOT rule is overly aggressive in assigning a
spam score
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6196
--- Comment #6 from Sidney Markowitz <si...@sidney.com> 2009-09-11 16:01:53 PST ---
Created an attachment (id=4533)
--> (https://issues.apache.org/SpamAssassin/attachment.cgi?id=4533)
Tom's test case as an attachment
It is obvious that it is the 'localhost.' that triggers the rule, but I can't
come up with a combination of settings that for trusted_network and so on that
actually makes the rule trigger on that example. If you look at the headers,
you see that if mailtrap.fortressitx.com is trusted, then you trust the
Received header in which it gets the mail from 'localhost.' which means that
the next Received header that says that localhost received from
busyagentpro.com is trusted, and that is the HELO that the rule looks at.
Otherwise, if mailtrap.fortressitx.com is not listed as trusted, then you move
up another two Received headers to find the one that is used, still skipping
the 'localhost.'.
Perhaps the parsing of
Received: from busyagentpro.com (busyagentpro.com [65.98.14.130]) by localhost.
(TC-3.1.040); Fri, 11 Sep 2009 17:52:26 -0400
has changed between 3.2 and the 3.3 trunk that I am testing with. I don't have
a 3.2 test setup handy where I am right now to see.
Are you able to run Spamassassin on the same machine that is producing this
with the -t -D -L options to produce a debug log file and attach it to this bug
report (as an attachment, not pasted in a comment)?
If anyone has 3.2 handy to test, I'm attaching Tom's example after I cleaned up
the line wrap issues from the copy and paste from the comment. To test it, try
adding
trusted_networks 69.72.141.51.
to your local config, as I don't see how the rule can possibly trigger without
that.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6196] FH_HELO_ENDS_DOT rule is overly aggressive in assigning a
spam score
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6196
John Hardin <jh...@impsec.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |jhardin@impsec.org
--- Comment #1 from John Hardin <jh...@impsec.org> 2009-09-11 12:38:39 PST ---
(In reply to comment #0)
>
> One of the cPanel developers has weighed in on the issue in the aforementioned
> URL
The URL fell off your post.
> and believes that the rule in question
> scores matching headers far too heavily given that a FQDN technically is
> supposed to end in a dot in the first place.
As has been mentioned before elsewhere, SA is not a standards-compliance
auditing tool. The score is based on that pattern appearing in actual spams and
not appearing in actual hams, regardless of whether or not it is syntactically
valid. At the moment the corpora contain almost no ham having that pattern, so
it looks like a strong spam indicator and is thus scored rather heavily.
If you can attach some FP examples to this bug, they could be added to the
corpora.
The score generation process for 3.3.0 is underway right now; I don't know
whether your report made the cutoff for adjusting this rule's score or not -
Justin? Is this too late to address in the 3.3.0 release?
Your customer could also just fix the HELO that their MTA sends and omit the
trailing period... :)
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
DO NOT REPLY [Bug 6196] FH_HELO_ENDS_DOT rule is overly aggressive
in assigning a spam score
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6196
--- Comment #7 from Sidney Markowitz <si...@sidney.com> 2009-09-11 18:01:29 PDT ---
I was able to test with 3.2 and it makes no difference.
I still don't see how the rule can be triggered given the possibilities of
which Received header is the last trusted one and so I need the debug log that
shows how the bug is happening on that server. I did look at it more carefully,
and this explanation of what I see may be clearer:
The first line is ignored for the purpose of this rule.
The second line says that the mail was received from mailtrap.fortressitx.com,
so it is the machine that created the Received header in the next line. If that
host's ip address is not in trusted_network, then the next and all subsequent
headers can not be believed and so the rule shouldn't trigger. And it doesn't
in my tests.
If mailtrap.fortressitx.com is in trusted_network, then the next Received
header can be believed not to be forged. It says that mailtrap.fortressitx.com
received the mail from localhost, which is also trusted to not forge headers,
and that continues to the header that has a HELO of 'localhost.', which is
trusted not to be forged. That one says that the ip address of the machine that
sent the 'localhost.' HELO is that of mailtrap.fortressitx.com which is
trusted, and therefore the next Received header was created on that machine and
wasn't forged. That header says that the mail was received by busyagentpro.com
with a good HELO. That should be the HELO string that is used for the test when
mailtrap.fortressitx.com is in the trusted network, and that is what I see when
I test it that way.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6196] FH_HELO_ENDS_DOT rule is overly aggressive in assigning a
spam score
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6196
--- Comment #5 from Tom Pawlowski <tp...@fortressitx.com> 2009-09-11 14:56:07 PST ---
Certainly, Sidney:
Received: (qmail 4296 invoked from network); 11 Sep 2009 21:52:28 -0000
Received: from mailtrap.fortressitx.com (69.72.141.51)
by tracker.fortressitx.com with SMTP; Fri, 11 Sep 2009 17:52:28 -0400
Received: from localhost (localhost.localdomain [127.0.0.1])
by mailtrap.fortressitx.com (Postfix) with ESMTP id 1BDEBDE788
for <su...@dedicatednow.com>; Fri, 11 Sep 2009 17:52:28 -0400 (EDT)
Received: from mailtrap.fortressitx.com ([127.0.0.1])
by localhost (mailtrap.fortressitx.com [127.0.0.1]) (amavisd-maia, port 10024)
with LMTP id 02120-03 for <su...@dedicatednow.com>;
Fri, 11 Sep 2009 17:52:27 -0400 (EDT)
Received: from localhost. (mailtrap.fortressitx.com [69.72.141.51])
by mailtrap.fortressitx.com (Postfix) with SMTP id 95DF3DE787
for <su...@dedicatednow.com>; Fri, 11 Sep 2009 17:52:27 -0400 (EDT)
X-Original-Hostname: busyagentpro.com
Received: from busyagentpro.com (busyagentpro.com [65.98.14.130]) by localhost.
(TC-3.1.040); Fri, 11 Sep 2009 17:52:26 -0400
Received: from localhost ([127.0.0.1]:39175)
by busyagentpro.com with esmtpa (Exim 4.69)
(envelope-from <de...@peachtreerealtygroup.com>)
id 1MmE2N-0003VK-Q3
for support@dedicatednow.com; Fri, 11 Sep 2009 17:52:12 -0400
Received: from 65.98.0.130 ([65.98.0.130]) by 208.116.61.76 (Horde MIME
library) with HTTP; Fri, 11 Sep 2009 17:52:11 -0400
Message-ID: <20...@208.116.61.76>
Date: Fri, 11 Sep 2009 17:52:11 -0400
From: dednow@peachtreerealtygroup.com
To: support@dedicatednow.com
Subject: Test Email
MIME-Version: 1.0
Content-Type: text/plain;
charset=ISO-8859-1;
DelSp="Yes";
format="flowed"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
User-Agent: Internet Messaging Program (IMP) H3 (4.1.6)
X-AntiAbuse: This header was added to track abuse, please include it with any
abuse report
X-AntiAbuse: Primary Hostname - busyagentpro.com
X-AntiAbuse: Original Domain - dedicatednow.com
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain - peachtreerealtygroup.com
X-Source:
X-Source-Args:
X-Source-Dir:
X-Virus-Scanned: Maia Mailguard 1.0.2a
X-Spam-Status: No, hits=4.23 tagged_above=1 required=5 tests=AWL=-0.000,
FH_HELO_ENDS_DOT=3.02, SPF_NEUTRAL=1.21
X-Spam-Level: ****
This is a test.
===
It wouldn't be this block would it?
Received: from localhost. (mailtrap.fortressitx.com [69.72.141.51])
by mailtrap.fortressitx.com (Postfix) with SMTP id 95DF3DE787
for <su...@dedicatednow.com>; Fri, 11 Sep 2009 17:52:27 -0400 (EDT)
That would indicate a problem with the configuration of the recipient MTA,
correct?
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
Re: [Bug 6196] FH_HELO_ENDS_DOT rule is overly aggressive in
assigning a spam score
Posted by John Hardin <jh...@impsec.org>.
On Fri, 11 Sep 2009, Karsten Br�ckelmann wrote:
> On Fri, 2009-09-11 at 16:37 -0400, Tom Pawlowski wrote:
>> As for the URL, I did see your comment about it. It was more for the
>> benefit of John in case it wasn't rendering on his end for whatever
>> reason since he didn't see it the first time.
I said that because I don't yet have the habit of looking for the explicit
URL field and was looking for it in the text body. My bad, for that I
blush and apologize.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
USMC Rules of Gunfighting #2: Anything worth shooting is worth
shooting twice. Ammo is cheap. Your life is expensive.
-----------------------------------------------------------------------
Today: the 8th anniversary of 9/11
Re: [Bug 6196] FH_HELO_ENDS_DOT rule is overly aggressive in
assigning a spam score
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Fri, 2009-09-11 at 17:02 -0400, Tom Pawlowski wrote:
> > In that case, you should have at least added the sender, too, not only
> > John and me. ;) Anyway...
>
> I think I did originally; well, replied to the original email that was
> sent, and received a bounceback from the bugzilla-daemon account. So I
> replied by hand. Should the reply-to have to been to
> dev@spamassassin.apache.org ?
No, there is only one way to add comments to bugzilla -- the web front-
end. Each bugzilla notification includes a handy link to the bug it
discusses, but it doesn't accept comments by mail.
The dev@ list I added, cause that's a list for development related
discussion, and also gets a copy of each bugzilla post.
> > Tom, please accept my apology -- you just caught me in a bad moment with
> > that, and I vented over a penny before really thinking a second time. I
> > should know better.
>
> No biggie, lord knows I've been there more times than I can count.
> Hope your evening gets a little less annoying.
>
> Take care,
> --T
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: [Bug 6196] FH_HELO_ENDS_DOT rule is overly aggressive in
assigning a spam score
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Fri, 2009-09-11 at 16:37 -0400, Tom Pawlowski wrote:
> I mistakenly thought replying via email would also update the web
> frontend, as a lot ticketing systems do. When you requested I add my
> reply to the comment field on the Bugzilla site, I did just that, so
> my original reply was resubmitted.
In that case, you should have at least added the sender, too, not only
John and me. ;) Anyway...
> As for the URL, I did see your comment about it. It was more for the
> benefit of John in case it wasn't rendering on his end for whatever
> reason since he didn't see it the first time. Is that really something
> to throw a hissy fit over?
Tom, please accept my apology -- you just caught me in a bad moment with
that, and I vented over a penny before really thinking a second time. I
should know better.
> 2009/9/11 Karsten Bräckelmann <gu...@rudersport.de>:
[...]
> > Sorry, really couldn't resist the urge to vent. Back to my cave now.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: [Bug 6196] FH_HELO_ENDS_DOT rule is overly aggressive in
assigning a spam score
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
Yes, text MAY be inline indeed. It is not only the top-posted crap that
counts.
On Fri, 2009-09-11 at 13:16 -0700, bugzilla-daemon wrote:
> https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6196
> --- Comment #3 from Tom Pawlowski <tp...@fortressitx.com> 2009-09-11 13:16:07 PST ---
> My apologies, here's the URL:
> http://forums.cpanel.net/f43/fh_helo_ends_dot-118261.html
Awesome. So you did not understand what I said in comment 2, that you
DID add the URL correctly -- despite the fact you even Cc'ed me on the
private reply, so you did read that comment.
And apparently, you didn't got it either when I stressed that fact again
on my reply telling you to add the comment to the bug report, instead of
replying by mail. Well, at least the most important fact you did read.
Cause I top-posted that? *sigh*
Sorry, really couldn't resist the urge to vent. Back to my cave now.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
[Bug 6196] FH_HELO_ENDS_DOT rule is overly aggressive in assigning a
spam score
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6196
Tom Pawlowski <tp...@fortressitx.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |tpawlowski@fortressitx.com
--- Comment #3 from Tom Pawlowski <tp...@fortressitx.com> 2009-09-11 13:16:07 PST ---
My apologies, here's the URL:
http://forums.cpanel.net/f43/fh_helo_ends_dot-118261.html
The thing is, from what I can see, the MTA isn't sending a hostname
with a period at the end. That was my original issue that I brought up
to the cPanel developers before Chris brought up the scoring criteria
and suggested mentioning it:
tom@chthonic:~$ telnet 208.116.61.76 25
Trying 208.116.61.76...
Connected to 208.116.61.76.
Escape character is '^]'.
220-busyagentpro.com ESMTP Exim 4.69 #1 Fri, 11 Sep 2009 15:58:07 -0400
220-We do not authorize the use of this system to transport unsolicited,
220 and/or bulk e-mail.
HELO
250 busyagentpro.com Hello [65.98.0.130]
Unless it's referring to the RDNS entry:
tom@chthonic:~$ dig +short -x 208.116.61.76
admin.busyagentpro.com.
tom@chthonic:~$ host 208.116.61.76
76.61.116.208.in-addr.arpa domain name pointer admin.busyagentpro.com.
But it needs to have that trailing dot to be a valid zone entry, so I
wouldn't think it's that.
Is the receiving MTA interpreting it as having a trailing dot because
it doesn't have the hostname format of mail.busyagentpro.com or the
like?
Thanks again for looking into this. It's a bit confounding and I
haven't been able to find anything specifically addressing it.
Regards,
Tom Pawlowski
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6196] FH_HELO_ENDS_DOT rule is overly aggressive in assigning a
spam score
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6196
Sidney Markowitz <si...@sidney.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |sidney@sidney.com
--- Comment #4 from Sidney Markowitz <si...@sidney.com> 2009-09-11 14:20:36 PST ---
Tom, could you attach an example email that triggers the rule? The dot at the
end of the FQDN from the reverse DNS should not cause a problem, and it doesn't
on other mail servers that I tried, and the HELO string of just the domain
doesn't cause a problem when I edit it in by hand on a sample email. So I'll
need an actual example with all headers, preferably with the SpamAssassin
result headers that prove that the rule is indeed triggering in that case.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6196] FH_HELO_ENDS_DOT rule is overly aggressive in assigning a
spam score
Posted by bu...@bugzilla.spamassassin.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6196
--- Comment #2 from Karsten Bräckelmann <gu...@rudersport.de> 2009-09-11 13:04:13 PST ---
> > One of the cPanel developers has weighed in on the issue in the aforementioned
> > URL
>
> The URL fell off your post.
See the URL field in the bug's information above.
> The score generation process for 3.3.0 is underway right now; I don't know
> whether your report made the cutoff for adjusting this rule's score or not -
> Justin? Is this too late to address in the 3.3.0 release?
It is too late to adjust the rule, since the mass-check for 3.3.0 started
already. It is not too late to manually limit the score, 'cause the GA run will
start after all mass-checks are done.
However, let's first see what the mass-checks and the GA based on that is going
to say about future scoring...
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.