You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by bu...@issues.apache.org on 2010/07/02 15:37:15 UTC
[Bug 6460] New: False positives with RCVD_ILLEGAL_IP
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
Summary: False positives with RCVD_ILLEGAL_IP
Product: Spamassassin
Version: 3.3.1
Platform: PC
OS/Version: Windows XP
Status: NEW
Severity: normal
Priority: P2
Component: Rules
AssignedTo: dev@spamassassin.apache.org
ReportedBy: prandal@herefordshire.gov.uk
The rule RCVD_ILLEGAL_IP seems to fire erroneously
>From debug:
Jul 2 14:28:07.473 [9951] dbg: metadata: X-Spam-Relays-Untrusted: [
ip=80.249.108.233 rdns=233.ncuk.mta.dotmailer.co.uk
helo=233.ncuk.mta.dotmailer.co.uk by=mx1.herefordshire.gov.uk ident= envfrom=
intl=0 id=o62DG6UR005877 auth= msa=0 ] [ ip=10.0.12.33 rdns=dmdroid201
helo=dmdroid201 by=233.ncuk.mta.dotmailer.co.uk ident=
envfrom=bo-BS7-6WPZ-1O40MT-BYTT8@itsmfnews.co.uk intl=0 id=h5nj4g0vbl8d auth=
msa=0 ]
The rule:
header RCVD_ILLEGAL_IP X-Spam-Relays-Untrusted =~ /
(?:by|ip)=(?:[05]|14|23|3[1679]|4[29]|1(?:0[0-6]|7[679]|8[15])|2(?:2[3-9]|[3-5]\d)|169\.254|192\.0\.2|198\.51\.100|203\.0\.113)\./
As the score for this is 3.399, it is creating a few too many false positives
here.
(sa-update reposrts version 953612)
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] [review] RCVD_ILLEGAL_IP false positives with
numerically-named relays
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
--- Comment #16 from Warren Togami <wt...@gmail.com> 2010-12-26 01:45:13 UTC ---
What will it take to make this go into 3.3 sa-updates?
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] [review] RCVD_ILLEGAL_IP false positives with
numerically-named relays
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
Mark Martinec <Ma...@ijs.si> changed:
What |Removed |Added
----------------------------------------------------------------------------
Blocks| |6467
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] [review] RCVD_ILLEGAL_IP false positives with
numerically-named relays
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
Mark Martinec <Ma...@ijs.si> changed:
What |Removed |Added
----------------------------------------------------------------------------
Blocks| |6516
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] [review] RCVD_ILLEGAL_IP false positives with
numerically-named relays
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
--- Comment #11 from Kevin A. McGrail <km...@pccc.com> 2010-11-30 18:31:45 UTC ---
(In reply to comment #10)
> Created an attachment (id=4828)
--> (https://issues.apache.org/SpamAssassin/attachment.cgi?id=4828) [details]
> Remove newly allocated and to-be-allocated IANA address ranges from the rule
>
> As it turns out, even now the rule triggers on several newly-allocated
> address ranges, causing false positives.
>
> The patch strips off all already allocated and to-be-allocated
> address ranges from the rule, leaving only the:
> - 0.0.0.0/8 self-identification network
> - multicast addresses
> - all three 'test' networks
AXB's comment was +1 for private use networks which doesn't make sense to me
since theoretically those should always be trusted, yes?
Since your rule doesn't hit 192.168.X or 10.X, I think you covered what makes
sense to me.
+1 KAM
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] RCVD_ILLEGAL_IP false positives with numerically-named
relays
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
Karsten Bräckelmann <gu...@rudersport.de> changed:
What |Removed |Added
----------------------------------------------------------------------------
Summary|False positives with |RCVD_ILLEGAL_IP false
|RCVD_ILLEGAL_IP |positives with
| |numerically-named relays
Status Whiteboard| |needs 2 votes for 3.3
--- Comment #5 from Karsten Bräckelmann <gu...@rudersport.de> 2010-07-23 11:05:16 EDT ---
Went ahead with the first variant, which is more elegant IMHO. Committed to
trunk, along with the fix for bug 6465. Please vote for 3.3.
Ensure RCVD_ILLEGAL_IP only matches on IPs, bug 6460. Remove 169.254/16
assigned as "link local" block, bug 6465.
Sending rules/20_head_tests.cf
Transmitting file data .
Committed revision 967120.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] [review] RCVD_ILLEGAL_IP false positives with
numerically-named relays
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
Mark Martinec <Ma...@ijs.si> changed:
What |Removed |Added
----------------------------------------------------------------------------
Priority|P2 |P1
Severity|normal |critical
--- Comment #15 from Mark Martinec <Ma...@ijs.si> 2010-12-04 18:59:35 UTC ---
(In reply to comment #12)
> trunk:
> Bug 6460: RCVD_ILLEGAL_IP false positives
> Sending rules/20_head_tests.cf
> Committed revision 1040826.
>
> I keep forgetting: is the rules directory now supposed to be
> common between branches? If so, I wonder why Günther's previous
> fix to this rule is only seen in trunk.
Apparently the 3.3 rules are now decoupled from trunk rules !?
Ending this frustration with the RCVD_ILLEGAL_IP rule misfiring
by manually backporting this rule change to 3.3:
3.3:
Bug 6460: RCVD_ILLEGAL_IP false positives
Sending rules/20_head_tests.cf
Committed revision 1042265.
If this change ends up in 3.3 sa-updates, this bug report can be closed,
otherwise it should be investigated as soon as possible.
Bumping up priority.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] [review] RCVD_ILLEGAL_IP false positives with
numerically-named relays
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
Mark Martinec <Ma...@ijs.si> changed:
What |Removed |Added
----------------------------------------------------------------------------
Target Milestone|Undefined |3.3.2
Summary|RCVD_ILLEGAL_IP false |[review] RCVD_ILLEGAL_IP
|positives with |false positives with
|numerically-named relays |numerically-named relays
Status Whiteboard|needs 2 votes for 3.3 |needs 1 vote for 3.3
--- Comment #6 from Mark Martinec <Ma...@ijs.si> 2010-11-30 14:34:23 UTC ---
> Went ahead with the first variant, which is more elegant IMHO. Committed to
> trunk, along with the fix for bug 6465. Please vote for 3.3.
> Ensure RCVD_ILLEGAL_IP only matches on IPs, bug 6460. Remove 169.254/16
> assigned as "link local" block, bug 6465.
+1
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] False positives with RCVD_ILLEGAL_IP
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
Phil Randal <ph...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |phil.randal@gmail.com
--- Comment #3 from Phil Randal <ph...@gmail.com> 2010-07-14 15:54:20 EDT ---
Latest mass checks:
RCVD_ILLEGAL_IP: bad, avg S/O=0.67 avg Spam%=0.11 avg Ham%=0.05
It will do badly until it is fixed...
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] False positives with RCVD_ILLEGAL_IP
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
Phil Randal <pr...@herefordshire.gov.uk> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |prandal@herefordshire.gov.u
| |k
OS/Version|Windows XP |Linux
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] False positives with RCVD_ILLEGAL_IP
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
--- Comment #2 from Phil Randal <pr...@herefordshire.gov.uk> 2010-07-02 10:28:57 EDT ---
(In reply to comment #1)
> Hmm, I'm seeing similar occasional FPs here with numerically-named mail relays.
> Example:
>
> dbg: metadata: X-Spam-Relays-Untrusted: [ ip=213.186.36.34
> rdns=5.mail-out.ovh.net helo=5.mail-out.ovh.net by=extranet.tradoc.fr ident=
> envfrom= intl=0 id=79A4E334033 auth= msa=0 ] [ ip=213.186.33.62 rdns=a2.ovh.net
> helo=mozg.ha.ovh.net by=5.mail-out.ovh.net ident= envfrom= intl=0 id= auth=
> msa=0 ]
> dbg: rules: ran header rule RCVD_ILLEGAL_IP ======> got hit: " by=5."
>
>
> I'd suggest that the regex needs to ensure that it is hitting on an IP address
> rather than a host name that happens to start with a number and a dot.
That's exactly it.
We've has 395 rule hits (on corpus of 55253) over the last 39.5 hours, ALL
false-positives.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] False positives with RCVD_ILLEGAL_IP
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
--- Comment #4 from Karsten Bräckelmann <gu...@rudersport.de> 2010-07-23 10:06:45 EDT ---
(In reply to comment #1)
> I'd suggest that the regex needs to ensure that it is hitting on an IP address
> rather than a host name that happens to start with a number and a dot.
Inserting a positive look-ahead after the equal in the RE would do, only
accepting an IP by ensuring a space as delimiter.
/(?=\d+\.\d+\.\d+\.\d+ )/
Another approach would be, to append a trailing part to the RE, enforcing only
IP style, non-hostname chars until the delimiting space.
/[0-9.]+ /
Devs, comments, thoughts?
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] [review] RCVD_ILLEGAL_IP false positives with
numerically-named relays
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
--- Comment #14 from AXB <al...@gmail.com> 2010-12-01 02:51:06 UTC ---
(In reply to comment #11)
> (In reply to comment #10)
> > Created an attachment (id=4828)
--> (https://issues.apache.org/SpamAssassin/attachment.cgi?id=4828) [details]
[details]
> > Remove newly allocated and to-be-allocated IANA address ranges from the rule
> >
> > As it turns out, even now the rule triggers on several newly-allocated
> > address ranges, causing false positives.
> >
> > The patch strips off all already allocated and to-be-allocated
> > address ranges from the rule, leaving only the:
> > - 0.0.0.0/8 self-identification network
> > - multicast addresses
> > - all three 'test' networks
>
> AXB's comment was +1 for private use networks which doesn't make sense to me
> since theoretically those should always be trusted, yes?
>
> Since your rule doesn't hit 192.168.X or 10.X, I think you covered what makes
> sense to me.
>
> +1 KAM
you're right, should have specified:
+1 not including non allocated IANA space in the rule.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] [review] RCVD_ILLEGAL_IP false positives with
numerically-named relays
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
--- Comment #12 from Mark Martinec <Ma...@ijs.si> 2010-11-30 18:37:23 UTC ---
trunk:
Bug 6460: RCVD_ILLEGAL_IP false positives
Sending rules/20_head_tests.cf
Committed revision 1040826.
I keep forgetting: is the rules directory now supposed to be
common between branches? If so, I wonder why Günther's previous
fix to this rule is only seen in trunk.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] [review] RCVD_ILLEGAL_IP false positives with
numerically-named relays
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
--- Comment #8 from Mark Martinec <Ma...@ijs.si> 2010-11-30 14:52:52 UTC ---
> This regex has gotten so complex I am wondering if a test case should be
> created for it?
Maybe it should just be dropped or restricted to private address space,
considering that IANA Unallocated Address Pool Exhaustion is estimated
for March 2011, and considering the 'fast' deployment rate of SA in
various distributions. The next allocation blocks released are about
to happen at a steady rate, so the rule is just waiting for its next
round of false positives.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] [review] RCVD_ILLEGAL_IP false positives with
numerically-named relays
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
Mark Martinec <Ma...@ijs.si> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution| |FIXED
--- Comment #17 from Mark Martinec <Ma...@ijs.si> 2010-12-28 09:27:40 UTC ---
> What will it take to make this go into 3.3 sa-updates?
As far as I can tell, the fix is already in 3.3 and 3.4 updates.
Closing. Please reopen if someone thinks otherwise.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] False positives with RCVD_ILLEGAL_IP
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
John Wilcock <jo...@tradoc.fr> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |john@tradoc.fr
--- Comment #1 from John Wilcock <jo...@tradoc.fr> 2010-07-02 10:11:09 EDT ---
Hmm, I'm seeing similar occasional FPs here with numerically-named mail relays.
Example:
dbg: metadata: X-Spam-Relays-Untrusted: [ ip=213.186.36.34
rdns=5.mail-out.ovh.net helo=5.mail-out.ovh.net by=extranet.tradoc.fr ident=
envfrom= intl=0 id=79A4E334033 auth= msa=0 ] [ ip=213.186.33.62 rdns=a2.ovh.net
helo=mozg.ha.ovh.net by=5.mail-out.ovh.net ident= envfrom= intl=0 id= auth=
msa=0 ]
dbg: rules: ran header rule RCVD_ILLEGAL_IP ======> got hit: " by=5."
I'd suggest that the regex needs to ensure that it is hitting on an IP address
rather than a host name that happens to start with a number and a dot.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] [review] RCVD_ILLEGAL_IP false positives with
numerically-named relays
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
--- Comment #10 from Mark Martinec <Ma...@ijs.si> 2010-11-30 18:23:17 UTC ---
Created an attachment (id=4828)
--> (https://issues.apache.org/SpamAssassin/attachment.cgi?id=4828)
Remove newly allocated and to-be-allocated IANA address ranges from the rule
As it turns out, even now the rule triggers on several newly-allocated
address ranges, causing false positives.
The patch strips off all already allocated and to-be-allocated
address ranges from the rule, leaving only the:
- 0.0.0.0/8 self-identification network
- multicast addresses
- all three 'test' networks
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] [review] RCVD_ILLEGAL_IP false positives with
numerically-named relays
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
--- Comment #13 from Mark Martinec <Ma...@ijs.si> 2010-11-30 18:40:40 UTC ---
> AXB's comment was +1 for private use networks which doesn't make sense to me
> since theoretically those should always be trusted, yes?
Yes, sorry, my fault, my misleading statement. The rule has nothing to do
with private networks. It covers self, multicast and test networks, and
previously also covered unallocated IANA networks, which are going away
in the following months.
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] [review] RCVD_ILLEGAL_IP false positives with
numerically-named relays
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
Kevin A. McGrail <km...@pccc.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |kmcgrail@pccc.com
--- Comment #7 from Kevin A. McGrail <km...@pccc.com> 2010-11-30 14:44:07 UTC ---
(In reply to comment #6)
> > Went ahead with the first variant, which is more elegant IMHO. Committed to
> > trunk, along with the fix for bug 6465. Please vote for 3.3.
> > Ensure RCVD_ILLEGAL_IP only matches on IPs, bug 6460. Remove 169.254/16
> > assigned as "link local" block, bug 6465.
>
> +1
This regex has gotten so complex I am wondering if a test case should be
created for it?
I can't really read it fast enough anymore to know what it should and shouldn't
be hitting on to vote without writing a test stub.
Regards,
KAM
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] [review] RCVD_ILLEGAL_IP false positives with
numerically-named relays
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
Warren Togami <wt...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |wtogami@gmail.com
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
[Bug 6460] [review] RCVD_ILLEGAL_IP false positives with
numerically-named relays
Posted by bu...@issues.apache.org.
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6460
AXB <al...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |alex.uribl@gmail.com
--- Comment #9 from AXB <al...@gmail.com> 2010-11-30 16:33:33 UTC ---
(In reply to comment #8)
> > This regex has gotten so complex I am wondering if a test case should be
> > created for it?
>
> Maybe it should just be dropped or restricted to private address space,
> considering that IANA Unallocated Address Pool Exhaustion is estimated
> for March 2011, and considering the 'fast' deployment rate of SA in
> various distributions. The next allocation blocks released are about
> to happen at a steady rate, so the rule is just waiting for its next
> round of false positives.
+1 to restricted to private address space only
--
Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.