You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Dianne Skoll <df...@roaringpenguin.com> on 2015/04/20 20:04:07 UTC
FPs on RCVD_ILLEGAL_IP
Hi,
Not sure if this is still an issue in 3.4, but I'm seeing tons of
FPs on RCVD_ILLEGAL_IP. Why? Because Microsoft (damn it to hell)
has started using RESERVED IP ranges internally! Have a look:
Received: from BLUPR10MB0835.namprd10.prod.outlook.com (0.163.216.13)
by BLUPR10MB0835.namprd10.prod.outlook.com (0.163.216.13)
with Microsoft SMTP Server (TLS) id 15.1.136.25;
Mon, 20 Apr 2015 17:43:48 +0000
Is anyone else seeing a sudden uptick in RCVD_ILLEGAL_IP FPs?
Regards,
Dianne.
Re: FPs on RCVD_ILLEGAL_IP
Posted by Axb <ax...@gmail.com>.
On 04/21/2015 01:13 AM, shanew@shanew.net wrote:
<snippppppppppppppppppp>
> I'm so glad to finally see this mentioned on here, because I was
> starting to doubt my own gut reaction that putting invalid IP
> addresses in Received is all sorts of broken. We noticed it last week
> after someone from Microsoft mentioned getting a rejection from our
> server, and looking back the first examples I was able to find of this
> was from Apr. 6. Before that emails following similar paths through
> Microsoft servers weren't doing this.
>
> I'm also happy to know there's some discussion going on with MS.
> When I mentioned it to an MS friend of mine last week he didn't seem
> particularly shocked that the "internal" headers wouldn't comply with
> expectations, but he also seemed surprised that anyone was looking at
> such headers as a way of determining spam. Hopefully MS will take
> this seriously, but I'm not holding my breath.
check latest headers - you may see the "UK Ministry of Defence" :-)
Re: FPs on RCVD_ILLEGAL_IP
Posted by Reindl Harald <h....@thelounge.net>.
Am 21.04.2015 um 16:21 schrieb Reindl Harald:
> Am 21.04.2015 um 15:59 schrieb Benny Pedersen:
>> Mark Martinec skrev den 2015-04-21 14:08:
>>> In any case, I think that RCVD_ILLEGAL_IP should not be adding
>>> score points if it sees an 0.0.0.0/8 address in a Received header field.
>>
>> why not add it to internal_networks in local.cf ?, why is spamassassin
>> only have 127.0.0.1 ?, is spamassassin at fault when it comes to iana
>> ipv4 listnings, and i belive there is listnings for ipv6 aswell
>>
>> my own rule of thumps is if you cant connect back to some ips, dont
>> accept email from it
>
> that is nonsense
>
> a 100% legit mail can made it through several hops for whatever reason
> and so have 192.168.0.1, 192.168.0.1 and 1921.68.0.2 in the Received
> headers - so you would reject that mails for *what* reason?
>
> because you can't connect to the middle hop in the local network where
> the message is comming from?
and BTW your "if you cant connect back to some ips, dont accept email
from it" is *in general* nonsense - many well planned mail systems have
completly different machines for incoming and outbound mail flow and so
you are not supposed to conenct back on port 25 by defintion
frankly you have no business at all to connect back because the outgoing
mailsystem is *not* the MX record for the sending domain
so please stop propose logic which only works in a small setup for a
guy, his wife and his brother
Re: FPs on RCVD_ILLEGAL_IP
Posted by Reindl Harald <h....@thelounge.net>.
Am 21.04.2015 um 15:59 schrieb Benny Pedersen:
> Mark Martinec skrev den 2015-04-21 14:08:
>> In any case, I think that RCVD_ILLEGAL_IP should not be adding
>> score points if it sees an 0.0.0.0/8 address in a Received header field.
>
> why not add it to internal_networks in local.cf ?, why is spamassassin
> only have 127.0.0.1 ?, is spamassassin at fault when it comes to iana
> ipv4 listnings, and i belive there is listnings for ipv6 aswell
>
> my own rule of thumps is if you cant connect back to some ips, dont
> accept email from it
that is nonsense
a 100% legit mail can made it through several hops for whatever reason
and so have 192.168.0.1, 192.168.0.1 and 1921.68.0.2 in the Received
headers - so you would reject that mails for *what* reason?
because you can't connect to the middle hop in the local network where
the message is comming from?
Re: FPs on RCVD_ILLEGAL_IP
Posted by Reindl Harald <h....@thelounge.net>.
Am 21.04.2015 um 16:26 schrieb Benny Pedersen:
> RW skrev den 2015-04-21 16:11:
>
>>> > In any case, I think that RCVD_ILLEGAL_IP should not be adding
>>> > score points if it sees an 0.0.0.0/8 address in a Received header
>>> > field.
>>>
>>> why not add it to internal_networks in local.cf ?,
>>
>> because internal_networks has no effect in the untrusted network.
>
> so in both internal_networks trusted_networks?
nosense - a random received header don't make a message to internal nor
to trusted
you don't want to hit "score ALL_TRUSTED -1.000" because of a random
header in the middle of a message just because "score RCVD_ILLEGAL_IP
1.3" becaus ein that case you can go into your config and add "score
RCVD_ILLEGAL_IP 0.3"
Re: FPs on RCVD_ILLEGAL_IP
Posted by Benny Pedersen <me...@junc.eu>.
RW skrev den 2015-04-21 16:11:
>> > In any case, I think that RCVD_ILLEGAL_IP should not be adding
>> > score points if it sees an 0.0.0.0/8 address in a Received header
>> > field.
>>
>> why not add it to internal_networks in local.cf ?,
>
> because internal_networks has no effect in the untrusted network.
so in both internal_networks trusted_networks ?
Re: FPs on RCVD_ILLEGAL_IP
Posted by RW <rw...@googlemail.com>.
On Tue, 21 Apr 2015 15:59:54 +0200
Benny Pedersen wrote:
> Mark Martinec skrev den 2015-04-21 14:08:
> > In any case, I think that RCVD_ILLEGAL_IP should not be adding
> > score points if it sees an 0.0.0.0/8 address in a Received header
> > field.
>
> why not add it to internal_networks in local.cf ?,
because internal_networks has no effect in the untrusted network.
Re: FPs on RCVD_ILLEGAL_IP
Posted by Benny Pedersen <me...@junc.eu>.
Mark Martinec skrev den 2015-04-21 14:08:
> In any case, I think that RCVD_ILLEGAL_IP should not be adding
> score points if it sees an 0.0.0.0/8 address in a Received header
> field.
why not add it to internal_networks in local.cf ?, why is spamassassin
only have 127.0.0.1 ?, is spamassassin at fault when it comes to iana
ipv4 listnings, and i belive there is listnings for ipv6 aswell
my own rule of thumps is if you cant connect back to some ips, dont
accept email from it, eg if a mx is only have 127.0.0.1, reject in
postfix, why should spamassassin be completely diffrent ?
is there bad side effects of list non routeble ips in local.cf ?
or what about trusted_networks, msa_networks ?
spamassassin is only as good as the rules it have
Re: FPs on RCVD_ILLEGAL_IP
Posted by Reindl Harald <h....@thelounge.net>.
Am 21.04.2015 um 21:23 schrieb shanew@shanew.net:
> On Tue, 21 Apr 2015, Dianne Skoll wrote:
>
>> On Tue, 21 Apr 2015 16:56:48 +0200
>> Matus UHLAR - fantomas <uh...@fantomas.sk> wrote:
>>
>>> what if Microsoft starts using other IP range tested by
>>> RCVD_ILLEGAL_IP?
>>
>> Then it deserves what it gets. Market forces are intended to penalize
>> companies that do stupid things and if we interfere in those market
>> forces, it will only encourage more stupid things.
>>
>> Or you could look at it this way: RCVD_ILLEGAL_IP was a really good
>> spam indicator until Microsoft messed up, so by using those IPs Microsoft
>> is helping spammers by forcing spam-fighters to reduce or abandon a
>> pretty good rule. Should that sort of behavior be rewarded?
>
> I presume detecting forged Received headers was the point of this rule
> all along, so if we all toss this rule out the window (or adjust to
> exclude this edge case), aren't we potentially encouraging spammers to
> "hide" their true networks in the same way?
frankly i don't care for *internal hops* but for the IP delivering mail
which can not be forged
> It occurs to me that if MS are the only people who are doing this, a
> meta-rule could counteract the score in that specific case. If it
> gets used much beyond that by legitimate actors though, that's a whole
> other story
looks like you are new on this list otherwise you would know that not so
long ago Yahoo did hit the same rule
my "problem" with that rule is that it hits practically no spam but only
ham and so it goes in the wrong direction entirely and the same happened
in the *ther direction* with "RP_MATCHES_RCVD" which is now
"T_RP_MATCHES_RCVD" and so no longer has impact for people not aware
both are rules doing nothing good
Re: FPs on RCVD_ILLEGAL_IP
Posted by Benny Pedersen <me...@junc.eu>.
Mark Martinec skrev den 2015-04-22 02:17:
> ... although there's a funny twist there. Some of these illegal
> IP addresses are not really a claimed-to-be IP address of a mailer,
> but come from an embedded e-mail address in a comment:
>
> Received: from unknown (HELO localhost)
> (jennifer_pride@sbcglobal.net@236.192.200.84)
> by mm-36-150-122-178.brest.dynamic.pppoe.byfly.by with ESMTPA;
> Tue, 21 Apr 2015 23:55:53 +0300
http://rbls.org/236.139.213.194
http://rbls.org/mm-36-150-122-178.brest.dynamic.pppoe.byfly.by
> Received: from unknown (HELO localhost)
> (bsobolewski@stockton-house.com@236.139.213.194)
> by 76.172.150.91 with ESMTPA; Tue, 21 Apr 2015 11:41:10 -0800
http://rbls.org/76.172.150.91
still possible to do smtp auth from this, all well
> so by a lucky coincidence a misparsed Received ends up
> yielding a useful-but-wrong result.
plain text auth over pppoe with pbl listed ip ?
why is is not using ESMTPSA, silly :=)
users trust there pppoe to be tcpdump safe, hmm
Re: FPs on RCVD_ILLEGAL_IP
Posted by Dianne Skoll <df...@roaringpenguin.com>.
On Wed, 22 Apr 2015 02:17:00 +0200
Mark Martinec <Ma...@ijs.si> wrote:
> Received: from unknown (HELO localhost)
> (bsobolewski@stockton-house.com@236.139.213.194)
> by 76.172.150.91 with ESMTPA; Tue, 21 Apr 2015 11:41:10 -0800
> so by a lucky coincidence a misparsed Received ends up
> yielding a useful-but-wrong result.
It's not really a mis-parsed Received so much as SpamAssassin trying
to cope with the myriad non-standard ways different MTAs put the
connecting IP in the Received: header. I've had to write
Received-parsing code and collected Received: headers from tons of
messages in the wild; there is a ridiculous number of variants.
Regards,
Dianne.
Re: FPs on RCVD_ILLEGAL_IP
Posted by Mark Martinec <Ma...@ijs.si>.
Dianne Skoll wrote:
> Mark Martinec <Ma...@ijs.si> wrote:
>> I can only conclude that a rule like RCVD_ILLEGAL_IP will hit
>> mostly on misconfigured or misguided sending mailers, not primarily
>> on spam.
>
> I disagree. Now that the Microsoft issue has been fixed, well over 95%
> of the mail on our system that hits RCVD_ILLEGAL_IP is spam.
You are right, I checked our logs and the RCVD_ILLEGAL_IP does
indeed mostly hit on spam.
... although there's a funny twist there. Some of these illegal
IP addresses are not really a claimed-to-be IP address of a mailer,
but come from an embedded e-mail address in a comment:
Received: from unknown (HELO localhost)
(jennifer_pride@sbcglobal.net@236.192.200.84)
by mm-36-150-122-178.brest.dynamic.pppoe.byfly.by with ESMTPA;
Tue, 21 Apr 2015 23:55:53 +0300
Received: from unknown (HELO localhost)
(bsobolewski@stockton-house.com@236.139.213.194)
by 76.172.150.91 with ESMTPA; Tue, 21 Apr 2015 11:41:10 -0800
so by a lucky coincidence a misparsed Received ends up
yielding a useful-but-wrong result.
Mark
Re: FPs on RCVD_ILLEGAL_IP
Posted by Dianne Skoll <df...@roaringpenguin.com>.
On Wed, 22 Apr 2015 00:47:56 +0200
Mark Martinec <Ma...@ijs.si> wrote:
> I can only conclude that a rule like RCVD_ILLEGAL_IP will hit
> mostly on misconfigured or misguided sending mailers, not primarily
> on spam.
I disagree. Now that the Microsoft issue has been fixed, well over 95%
of the mail on our system that hits RCVD_ILLEGAL_IP is spam.
Regards,
Dianne.
Re: FPs on RCVD_ILLEGAL_IP
Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 21 Apr 2015, at 18:47, Mark Martinec wrote:
> There is no benefit to spammers (and a likely disservice to them)
> for forging a non-trustworthy external Received header field
> and providing some unusual IP address there, and they cannot forge
> the boundary Received header field inserted by recipient's own mailer.
This is all true.
> I can only conclude that a rule like RCVD_ILLEGAL_IP will hit
> mostly on misconfigured or misguided sending mailers, not primarily
> on spam.
This would be true if the people and tools trying to investigate spam
sources AND spammers were uniformly (or even broadly) as smart about
email as you or as anyone else who has been working with email
intensively for many years.
That is not the case, as evidenced by the fact that RCVD_ILLEGAL_IP
actually has a history of being a very reliable test except for the
recent periods of Yahoo and Microsoft engaging in Stupid Freemail
Tricks. Spammers forge Received headers to send investigators & their
tools on wild goose chases, both because they don't understand the net
effects and because once in a while it works.
It is worth noting that I have a large handful of very reliable
SCC_RCVD_FORMAT_* custom rules, some of which date to 2003 yet still get
hits, because the same spammers and/or spamware have been creating
Received headers in patterns unlike any real MTA for a dozen years. When
they stop being useful, I'll consider the possibility that spammers are
not generally morons engaged in high-effort self-defeating tactics.
Re: FPs on RCVD_ILLEGAL_IP
Posted by Mark Martinec <Ma...@ijs.si>.
shanew wrote:
> I presume detecting forged Received headers was the point of this rule
> all along, so if we all toss this rule out the window (or adjust to
> exclude this edge case), aren't we potentially encouraging spammers to
> "hide" their true networks in the same way?
There is no benefit to spammers (and a likely disservice to them)
for forging a non-trustworthy external Received header field
and providing some unusual IP address there, and they cannot forge
the boundary Received header field inserted by recipient's own mailer.
I can only conclude that a rule like RCVD_ILLEGAL_IP will hit
mostly on misconfigured or misguided sending mailers, not primarily
on spam.
Reindl Harald wrote:
> my "problem" with that rule is that it hits practically no spam
> but only ham and so it goes in the wrong direction entirely
Most likely.
Mark
Re: FPs on RCVD_ILLEGAL_IP
Posted by Axb <ax...@gmail.com>.
On 04/21/2015 09:23 PM, shanew@shanew.net wrote:
> On Tue, 21 Apr 2015, Dianne Skoll wrote:
>
>> On Tue, 21 Apr 2015 16:56:48 +0200
>> Matus UHLAR - fantomas <uh...@fantomas.sk> wrote:
>>
>>> what if Microsoft starts using other IP range tested by
>>> RCVD_ILLEGAL_IP?
>>
>> Then it deserves what it gets. Market forces are intended to penalize
>> companies that do stupid things and if we interfere in those market
>> forces, it will only encourage more stupid things.
>>
>> Or you could look at it this way: RCVD_ILLEGAL_IP was a really good
>> spam indicator until Microsoft messed up, so by using those IPs Microsoft
>> is helping spammers by forcing spam-fighters to reduce or abandon a
>> pretty good rule. Should that sort of behavior be rewarded?
>
> I presume detecting forged Received headers was the point of this rule
> all along, so if we all toss this rule out the window (or adjust to
> exclude this edge case), aren't we potentially encouraging spammers to
> "hide" their true networks in the same way?
>
> It occurs to me that if MS are the only people who are doing this, a
> meta-rule could counteract the score in that specific case. If it
> gets used much beyond that by legitimate actors though, that's a whole
> other story.
afaik, the MS "issue" has been fixed.
Re: FPs on RCVD_ILLEGAL_IP
Posted by sh...@shanew.net.
On Tue, 21 Apr 2015, Dianne Skoll wrote:
> On Tue, 21 Apr 2015 16:56:48 +0200
> Matus UHLAR - fantomas <uh...@fantomas.sk> wrote:
>
>> what if Microsoft starts using other IP range tested by
>> RCVD_ILLEGAL_IP?
>
> Then it deserves what it gets. Market forces are intended to penalize
> companies that do stupid things and if we interfere in those market
> forces, it will only encourage more stupid things.
>
> Or you could look at it this way: RCVD_ILLEGAL_IP was a really good
> spam indicator until Microsoft messed up, so by using those IPs Microsoft
> is helping spammers by forcing spam-fighters to reduce or abandon a
> pretty good rule. Should that sort of behavior be rewarded?
I presume detecting forged Received headers was the point of this rule
all along, so if we all toss this rule out the window (or adjust to
exclude this edge case), aren't we potentially encouraging spammers to
"hide" their true networks in the same way?
It occurs to me that if MS are the only people who are doing this, a
meta-rule could counteract the score in that specific case. If it
gets used much beyond that by legitimate actors though, that's a whole
other story.
--
Public key #7BBC68D9 at | Shane Williams
http://pgp.mit.edu/ | System Admin - UT CompSci
=----------------------------------+-------------------------------
All syllogisms contain three lines | shanew@shanew.net
Therefore this is not a syllogism | www.ischool.utexas.edu/~shanew
Re: FPs on RCVD_ILLEGAL_IP
Posted by Dianne Skoll <df...@roaringpenguin.com>.
On Tue, 21 Apr 2015 16:56:48 +0200
Matus UHLAR - fantomas <uh...@fantomas.sk> wrote:
> what if Microsoft starts using other IP range tested by
> RCVD_ILLEGAL_IP?
Then it deserves what it gets. Market forces are intended to penalize
companies that do stupid things and if we interfere in those market
forces, it will only encourage more stupid things.
Or you could look at it this way: RCVD_ILLEGAL_IP was a really good
spam indicator until Microsoft messed up, so by using those IPs Microsoft
is helping spammers by forcing spam-fighters to reduce or abandon a
pretty good rule. Should that sort of behavior be rewarded?
Regards,
Dianne.
Re: FPs on RCVD_ILLEGAL_IP
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
>> > On 21.04.15 14:08, Mark Martinec wrote:
>> > >In any case, I think that RCVD_ILLEGAL_IP should not be adding
>> > >score points if it sees an 0.0.0.0/8 address in a Received header
>> > >field.
>> On Tue, 21 Apr 2015 15:56:27 +0200
>> Matus UHLAR - fantomas wrote:
>> > Why not? Should not it depend mostly on what masscheck say?
On 21.04.15 15:15, RW wrote:
>> The reason is that the FPs are on 0/8, which is only a small part of
>> the total address space.
>
>I meant, of the total address space that RCVD_ILLEGAL_IP currently
>hits.
what if Microsoft starts using other IP range tested by RCVD_ILLEGAL_IP?
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
42.7 percent of all statistics are made up on the spot.
Re: FPs on RCVD_ILLEGAL_IP
Posted by RW <rw...@googlemail.com>.
On Tue, 21 Apr 2015 15:10:09 +0100
RW wrote:
> On Tue, 21 Apr 2015 15:56:27 +0200
> Matus UHLAR - fantomas wrote:
>
> > On 21.04.15 14:08, Mark Martinec wrote:
> > >In any case, I think that RCVD_ILLEGAL_IP should not be adding
> > >score points if it sees an 0.0.0.0/8 address in a Received header
> > >field.
> >
> > Why not? Should not it depend mostly on what masscheck say?
>
>
> The reason is that the FPs are on 0/8, which is only a small part of
> the total address space.
I meant, of the total address space that RCVD_ILLEGAL_IP currently
hits.
Re: FPs on RCVD_ILLEGAL_IP
Posted by RW <rw...@googlemail.com>.
On Tue, 21 Apr 2015 15:56:27 +0200
Matus UHLAR - fantomas wrote:
> On 21.04.15 14:08, Mark Martinec wrote:
> >In any case, I think that RCVD_ILLEGAL_IP should not be adding
> >score points if it sees an 0.0.0.0/8 address in a Received header
> >field.
>
> Why not? Should not it depend mostly on what masscheck say?
The reason is that the FPs are on 0/8, which is only a small part of
the total address space. Without it the rule could still hit ~96% of
the spam, but would warrant a much higher score.
Re: FPs on RCVD_ILLEGAL_IP
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
On 21.04.15 14:08, Mark Martinec wrote:
>In any case, I think that RCVD_ILLEGAL_IP should not be adding
>score points if it sees an 0.0.0.0/8 address in a Received header field.
Why not? Should not it depend mostly on what masscheck say?
...some time ago, I noticed that Eset Smart Security (and later, Eset Antivirus
iirc) breaks mail bodies by inserting long (>76 character) lines
containing their signature (advertisement, including www.eset.com link)
to quoted-printable body, thus hitting MIME_QP_LONG_LINE.
Further, someone using their software apparently spammed and got their URL
to BLACKLIST, thus any ESET-ware checked mail also hit
Should we stop scoring on any possible rule just because false positives
can appear?
The BAYES should be able to fix individual FPS, and people at microft, eset
etc. should stop doing stupid things.
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
I just got lost in thought. It was unfamiliar territory.
Re: FPs on RCVD_ILLEGAL_IP
Posted by Mark Martinec <Ma...@ijs.si>.
In any case, I think that RCVD_ILLEGAL_IP should not be adding
score points if it sees an 0.0.0.0/8 address in a Received header field.
Mark
Re: FPs on RCVD_ILLEGAL_IP
Posted by RW <rw...@googlemail.com>.
On Tue, 21 Apr 2015 11:40:45 +0100
Martin Gregorie wrote:
> On Mon, 2015-04-20 at 21:15 -0400, Dianne Skoll wrote:
> > On Mon, 20 Apr 2015 17:02:09 -0700 (PDT)
> > John Hardin <jh...@impsec.org> wrote:
> >
> > > I suggest that this rule should treat 0/8 as equivalent to 127/8.
> > > That's essentially what it's reserved for, just "local to the LAN"
> > > vs. "local to the host".
> >
> > Does 0/8 really mean that? On at least one OS (Linux), the TCP
> > stack treats it specially:
> >
> The situation seems to be confused at best. Many of the lists of
> reserved IP ranges don't even mention 0/8.
See
http://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml
0/8 is relatively small block compared with the multicast and
reserved blocks which are both /4. If you assume that spammers are
hitting these addresses by accident then ~96% of the hits will be in
those two alone.
Re: FPs on RCVD_ILLEGAL_IP
Posted by Martin Gregorie <ma...@gregorie.org>.
On Mon, 2015-04-20 at 21:15 -0400, Dianne Skoll wrote:
> On Mon, 20 Apr 2015 17:02:09 -0700 (PDT)
> John Hardin <jh...@impsec.org> wrote:
>
> > I suggest that this rule should treat 0/8 as equivalent to 127/8.
> > That's essentially what it's reserved for, just "local to the LAN"
> > vs. "local to the host".
>
> Does 0/8 really mean that? On at least one OS (Linux), the TCP stack
> treats it specially:
>
The situation seems to be confused at best. Many of the lists of
reserved IP ranges don't even mention 0/8.
- the Wikipedia reserved IP page says its for multicasting on the LAN
and quotes RFC1700 page 4, where there's no mention of it that I
could see but its private address page doesn't mention 0/8 at all,
but a shed-load of other special addresses are also listed.
- The IANA IPv4 Address Space Registry says 0/8 for
'IANA - Local Identification'
- RFC 1918 defines the 10/8, 172.16/12 and 192.168/16 as private address
ranges but leaves it at that.
- Microsoft adds 169.254/16 as something called APIPA in addition to
RFC 1918. RFC 3927 describes it as link-local. Is this reserved for
asking a DHCP server for an address?
At this point I gave up the search and decided that 'confused' is a fair
description.
I hope this helps, but suspect it doesn't.
Martin
Re: FPs on RCVD_ILLEGAL_IP
Posted by Mark Martinec <Ma...@ijs.si>.
Dianne Skoll wrote:
> On Mon, 20 Apr 2015 17:02:09 -0700 (PDT)
> John Hardin <jh...@impsec.org> wrote:
>
>> I suggest that this rule should treat 0/8 as equivalent to 127/8.
>> That's essentially what it's reserved for, just "local to the LAN"
>> vs. "local to the host".
>
> Does 0/8 really mean that? On at least one OS (Linux), the TCP stack
> treats it specially:
>
> $ telnet 0.1.2.3
> Trying 0.1.2.3...
> telnet: Unable to connect to remote host: Invalid argument
>
> The EINVAL return is certainly not the same as trying a nonexistent
> host:
>
> $ telnet 10.11.12.13
> Trying 10.11.12.13...
> [hangs]
>
> I don't think 0/8 was intended for real traffic. I understood it to be
> intended only for hosts trying to discover their real IP addresses.
The 0.0.0.0/8 is an 'unspecified' address. In principle a host is
free to use it for whatever purposes internally (or for network
discovery),
but is not routable outside the host. In that sense it behaves the
same as 127.0.0.0/8 and I think for the purposes of seeing it in
a Received header field outside of the 'trusted' zone it should be
treated as any foreign private IP address space, i.e. it may be
valid for the host which inserted it, but has no value for us
the receiving party.
Btw, the same should apply to addresses ::/128 and ::1/128 .
Mark
Re: FPs on RCVD_ILLEGAL_IP
Posted by Dianne Skoll <df...@roaringpenguin.com>.
On Mon, 20 Apr 2015 17:02:09 -0700 (PDT)
John Hardin <jh...@impsec.org> wrote:
> I suggest that this rule should treat 0/8 as equivalent to 127/8.
> That's essentially what it's reserved for, just "local to the LAN"
> vs. "local to the host".
Does 0/8 really mean that? On at least one OS (Linux), the TCP stack
treats it specially:
$ telnet 0.1.2.3
Trying 0.1.2.3...
telnet: Unable to connect to remote host: Invalid argument
The EINVAL return is certainly not the same as trying a nonexistent
host:
$ telnet 10.11.12.13
Trying 10.11.12.13...
[hangs]
I don't think 0/8 was intended for real traffic. I understood it to be
intended only for hosts trying to discover their real IP addresses.
Regards,
Dianne.
Re: FPs on RCVD_ILLEGAL_IP
Posted by Mark Martinec <Ma...@ijs.si>.
John Hardin wrote:
> I suggest that this rule should treat 0/8 as equivalent to 127/8.
> That's essentially what it's reserved for, just "local to the LAN" vs.
> "local to the host".
I fully agree.
Mark
Re: FPs on RCVD_ILLEGAL_IP
Posted by John Hardin <jh...@impsec.org>.
On Mon, 20 Apr 2015, shanew@shanew.net wrote:
> I'm also happy to know there's some discussion going on with MS.
> When I mentioned it to an MS friend of mine last week he didn't seem
> particularly shocked that the "internal" headers wouldn't comply with
> expectations, but he also seemed surprised that anyone was looking at
> such headers as a way of determining spam.
Unfortunately it's rather difficult to determine whether the Received
headers external to the local network are in somebody else's local
network.
I suggest that this rule should treat 0/8 as equivalent to 127/8. That's
essentially what it's reserved for, just "local to the LAN" vs. "local to
the host".
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
USMC Rules of Gunfighting #2: Anything worth shooting is worth
shooting twice. Ammo is cheap. Your life is expensive.
-----------------------------------------------------------------------
3 days until Max Planck's 157th birthday
Re: FPs on RCVD_ILLEGAL_IP
Posted by sh...@shanew.net.
On Mon, 20 Apr 2015, Axb wrote:
> On 04/20/2015 08:04 PM, Dianne Skoll wrote:
>> Hi,
>>
>> Not sure if this is still an issue in 3.4, but I'm seeing tons of
>> FPs on RCVD_ILLEGAL_IP. Why? Because Microsoft (damn it to hell)
>> has started using RESERVED IP ranges internally! Have a look:
>>
>> Received: from BLUPR10MB0835.namprd10.prod.outlook.com (0.163.216.13)
>> by BLUPR10MB0835.namprd10.prod.outlook.com (0.163.216.13)
>> with Microsoft SMTP Server (TLS) id 15.1.136.25;
>> Mon, 20 Apr 2015 17:43:48 +0000
>>
>> Is anyone else seeing a sudden uptick in RCVD_ILLEGAL_IP FPs?
>
> There is an ongoing discussion about this with MS, thru backchannels.
>
> They're intentionally using the 0/8 to mask internal IPs.
> A very VERY bad choice and they have been advised that not only SA thinks
> it's a bad idea.
>
> Axb
I'm so glad to finally see this mentioned on here, because I was
starting to doubt my own gut reaction that putting invalid IP
addresses in Received is all sorts of broken. We noticed it last week
after someone from Microsoft mentioned getting a rejection from our
server, and looking back the first examples I was able to find of this
was from Apr. 6. Before that emails following similar paths through
Microsoft servers weren't doing this.
I'm also happy to know there's some discussion going on with MS.
When I mentioned it to an MS friend of mine last week he didn't seem
particularly shocked that the "internal" headers wouldn't comply with
expectations, but he also seemed surprised that anyone was looking at
such headers as a way of determining spam. Hopefully MS will take
this seriously, but I'm not holding my breath.
--
Public key #7BBC68D9 at | Shane Williams
http://pgp.mit.edu/ | System Admin - UT CompSci
=----------------------------------+-------------------------------
All syllogisms contain three lines | shanew@shanew.net
Therefore this is not a syllogism | www.ischool.utexas.edu/~shanew
Re: FPs on RCVD_ILLEGAL_IP
Posted by RW <rw...@googlemail.com>.
On Mon, 20 Apr 2015 20:50:08 +0200
Axb wrote:
> On 04/20/2015 08:04 PM, Dianne Skoll wrote:
> > Is anyone else seeing a sudden uptick in RCVD_ILLEGAL_IP FPs?
>
> There is an ongoing discussion about this with MS, thru backchannels.
>
> They're intentionally using the 0/8 to mask internal IPs.
> A very VERY bad choice and they have been advised that not only SA
> thinks it's a bad idea.
Yahoo did the same thing a couple of months.
Perhaps it would be useful to split RCVD_ILLEGAL_IP into a local and
a non-local version according to whether the IP addresses can be
routed outside of the local network.
The RCVD_ILLEGAL_IP hits I'm seeing on spam are mostly multicast
addresses.
Re: FPs on RCVD_ILLEGAL_IP
Posted by Axb <ax...@gmail.com>.
On 04/20/2015 08:04 PM, Dianne Skoll wrote:
> Hi,
>
> Not sure if this is still an issue in 3.4, but I'm seeing tons of
> FPs on RCVD_ILLEGAL_IP. Why? Because Microsoft (damn it to hell)
> has started using RESERVED IP ranges internally! Have a look:
>
> Received: from BLUPR10MB0835.namprd10.prod.outlook.com (0.163.216.13)
> by BLUPR10MB0835.namprd10.prod.outlook.com (0.163.216.13)
> with Microsoft SMTP Server (TLS) id 15.1.136.25;
> Mon, 20 Apr 2015 17:43:48 +0000
>
> Is anyone else seeing a sudden uptick in RCVD_ILLEGAL_IP FPs?
There is an ongoing discussion about this with MS, thru backchannels.
They're intentionally using the 0/8 to mask internal IPs.
A very VERY bad choice and they have been advised that not only SA
thinks it's a bad idea.
Axb
Re: FPs on RCVD_ILLEGAL_IP
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
I'm not finding the rule hitting very much here.
And it doesn't appear to be very high volume looking at
http://ruleqa.spamassassin.org/20150419-r1674595-n/RCVD_ILLEGAL_IP/detail but
the S/O is high.
Re: FPs on RCVD_ILLEGAL_IP
Posted by Reindl Harald <h....@thelounge.net>.
Am 20.04.2015 um 22:48 schrieb Axb:
> On 04/20/2015 09:03 PM, Reindl Harald wrote:
>> well, received headers in the middle of a message are not that good for
>> classification at all
>
> sez the expert..
well, i was victim of a appliance starting from one day to another deep
header inspection for RBL's as well as PTR and it was a nightmare
> look at 20_dnsbl_tests.cf and you'll see that not all lookups are
> lastexternal
hence i disabled any builtin RBL's and replaced them with custom rules
maintained from a webinterface
> or put the internet cafes on 41.203.69.0/24 in a local BL and see it
> catch 419's injection points.
these are on lastexternal RBL#s
> obviously you won't want to run deep header lookups against PBL or XBL
obviously i don't want to run *any* deep header lookups
> but injection points on VPNs, etc can only be detected through deep
> header parsing
RBL is lastexternal - anything else has to be caught by others rules,
bayes and URIBL, do what you want for your mailservers, my job is first
to deliver email and then filter out as much as possible spam with as
less as possible false poitives because 1 FP hurts much more than 20 not
caught junk messages
Re: FPs on RCVD_ILLEGAL_IP
Posted by Axb <ax...@gmail.com>.
On 04/20/2015 09:03 PM, Reindl Harald wrote:
> well, received headers in the middle of a message are not that good for
> classification at all
sez the expert..
look at 20_dnsbl_tests.cf and you'll see that not all lookups are
lastexternal
or put the internet cafes on 41.203.69.0/24 in a local BL and see it
catch 419's injection points.
obviously you won't want to run deep header lookups against PBL or XBL
but injection points on VPNs, etc can only be detected through deep
header parsing.
Re: FPs on RCVD_ILLEGAL_IP
Posted by Benny Pedersen <me...@junc.eu>.
Benny Pedersen skrev den 2015-04-20 21:34:
> John Hardin skrev den 2015-04-20 21:24:
>> On Mon, 20 Apr 2015, Reindl Harald wrote:
>>
>>> well, received headers in the middle of a message are not that good
>>> for classification at all
>>
>> It is if they are sloppily forged.
>
> good plan here https://dmarcian.com/spf-survey/outlock.com ipv6 only
> spf
ups typo https://dmarcian.com/spf-survey/outlook.com
but its still without 0/8 in spf, spf fail
Re: FPs on RCVD_ILLEGAL_IP
Posted by Reindl Harald <h....@thelounge.net>.
Am 20.04.2015 um 21:34 schrieb Benny Pedersen:
> John Hardin skrev den 2015-04-20 21:24:
>> On Mon, 20 Apr 2015, Reindl Harald wrote:
>>
>>> well, received headers in the middle of a message are not that good
>>> for classification at all
>>
>> It is if they are sloppily forged.
>
> good plan here https://dmarcian.com/spf-survey/outlock.com ipv6 only spf
completly off-topic, SPF is about the connecting IP and not a random
Received header in the middle which can com from *any* software and hop
a message touched until it hits your server
Re: FPs on RCVD_ILLEGAL_IP
Posted by Benny Pedersen <me...@junc.eu>.
John Hardin skrev den 2015-04-20 21:24:
> On Mon, 20 Apr 2015, Reindl Harald wrote:
>
>> well, received headers in the middle of a message are not that good
>> for classification at all
>
> It is if they are sloppily forged.
good plan here https://dmarcian.com/spf-survey/outlock.com ipv6 only spf
Re: FPs on RCVD_ILLEGAL_IP
Posted by John Hardin <jh...@impsec.org>.
On Mon, 20 Apr 2015, Reindl Harald wrote:
> well, received headers in the middle of a message are not that good for
> classification at all
It is if they are sloppily forged.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Your mouse has moved. Your Windows Operating System must be
relicensed due to this hardware change. Please contact Microsoft
to obtain a new activation key. If this hardware change results in
added functionality you may be subject to additional license fees.
Your system will now shut down. Thank you for choosing Microsoft.
-----------------------------------------------------------------------
3 days until Max Planck's 157th birthday
Re: FPs on RCVD_ILLEGAL_IP
Posted by Reindl Harald <h....@thelounge.net>.
Am 20.04.2015 um 20:59 schrieb Axb:
> On 04/20/2015 08:54 PM, Reindl Harald wrote:
>>
>> Am 20.04.2015 um 20:51 schrieb Axb:
>>> On 04/20/2015 08:45 PM, Reindl Harald wrote:
>>>>
>>>> looks like RCVD_ILLEGAL_IP does much more harm than good
>>>
>>> the rule is good - send your complaint to Microsoft.
>>> 0/8 is not assigned
>>
>> no a rule with 1.3 points hitting to 99.999% ham messages is not good
>> and it does not matter who is responsible - sening a complaint to
>> microsoft does not solve a *real problem now*
>>
>> the rule is good in a perfect world
>> mailservers have to interact with the real world
>
> 0/8 is not the real world
well, received headers in the middle of a message are not that good for
classification at all
> and it does indeed matter who is responsible
not for the rcpt of a legit mail
> lower the score and move on
i did since it hitted only *one* spam message in teh current month and
the rest was 100% sure ham
Re: FPs on RCVD_ILLEGAL_IP
Posted by Axb <ax...@gmail.com>.
On 04/20/2015 08:54 PM, Reindl Harald wrote:
>
> Am 20.04.2015 um 20:51 schrieb Axb:
>> On 04/20/2015 08:45 PM, Reindl Harald wrote:
>>>
>>> Am 20.04.2015 um 20:42 schrieb Kevin A. McGrail:
>>>> On 4/20/2015 2:23 PM, Dianne Skoll wrote:
>>>>> On Mon, 20 Apr 2015 14:20:35 -0400
>>>>> "Kevin A. McGrail" <KM...@PCCC.com> wrote:
>>>>>
>>>>>> Are you seeing it on a lot of emails?
>>>>> Over 25000 today; every single one of them from an "...outlook.com"
>>>>> server. :(
>>>>>
>>>>> Regards,
>>>>>
>>>>> Dianne.
>>>> Weird. Any chance you know one of the senders and can ask them to
>>>> email
>>>> kmcgrail@pccc.com and raptorreview15@pccc.com with a test? then you and
>>>> I can compare tests hit, etc.
>>>
>>> seems to not hit every mail from outlook.com but when it hits at all
>>> most are FP's with BAYES_00 and just *one* spam message here the whole
>>> month with a score of 14 and so catched by enough other rules
>>>
>>> looks like RCVD_ILLEGAL_IP does much more harm than good
>>
>> the rule is good - send your complaint to Microsoft.
>> 0/8 is not assigned
>
> no a rule with 1.3 points hitting to 99.999% ham messages is not good
> and it does not matter who is responsible - sening a complaint to
> microsoft does not solve a *real problem now*
>
> the rule is good in a perfect world
> mailservers have to interact with the real world
0/8 is not the real world - and it does indeed matter who is responsible.
lower the score and move on.
Re: FPs on RCVD_ILLEGAL_IP
Posted by Dianne Skoll <df...@roaringpenguin.com>.
On Mon, 20 Apr 2015 14:59:19 -0400
"Kevin A. McGrail" <KM...@PCCC.com> wrote:
> I don't show it hitting on ham on my system though I trust DFS and
> AXB's experience in this matter. You might want to score it to 0
> because I'm not going to raise a panic flag on a 1.3 score rule when
> Microsoft could come to their senses in a day.
Yes, agreed.
Also, I'd like to clarify what I wrote earlier. Although many valid
messages from Office 365 are hitting the rule, I don't mean to imply
that all of them were scored as spam. Most were not; in most cases,
the rule was not enough to cross the threshold. But it did happen often
enough that a trickle of customer complaints started coming in.
Regards,
Dianne.
Re: FPs on RCVD_ILLEGAL_IP
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 4/20/2015 2:54 PM, Reindl Harald wrote:
> no a rule with 1.3 points hitting to 99.999% ham messages is not good
> and it does not matter who is responsible - sening a complaint to
> microsoft does not solve a *real problem now*
I don't show it hitting on ham on my system though I trust DFS and AXB's
experience in this matter. You might want to score it to 0 because I'm
not going to raise a panic flag on a 1.3 score rule when Microsoft could
come to their senses in a day.
Regards,
KAM
Re: FPs on RCVD_ILLEGAL_IP
Posted by Reindl Harald <h....@thelounge.net>.
Am 20.04.2015 um 20:51 schrieb Axb:
> On 04/20/2015 08:45 PM, Reindl Harald wrote:
>>
>> Am 20.04.2015 um 20:42 schrieb Kevin A. McGrail:
>>> On 4/20/2015 2:23 PM, Dianne Skoll wrote:
>>>> On Mon, 20 Apr 2015 14:20:35 -0400
>>>> "Kevin A. McGrail" <KM...@PCCC.com> wrote:
>>>>
>>>>> Are you seeing it on a lot of emails?
>>>> Over 25000 today; every single one of them from an "...outlook.com"
>>>> server. :(
>>>>
>>>> Regards,
>>>>
>>>> Dianne.
>>> Weird. Any chance you know one of the senders and can ask them to email
>>> kmcgrail@pccc.com and raptorreview15@pccc.com with a test? then you and
>>> I can compare tests hit, etc.
>>
>> seems to not hit every mail from outlook.com but when it hits at all
>> most are FP's with BAYES_00 and just *one* spam message here the whole
>> month with a score of 14 and so catched by enough other rules
>>
>> looks like RCVD_ILLEGAL_IP does much more harm than good
>
> the rule is good - send your complaint to Microsoft.
> 0/8 is not assigned
no a rule with 1.3 points hitting to 99.999% ham messages is not good
and it does not matter who is responsible - sening a complaint to
microsoft does not solve a *real problem now*
the rule is good in a perfect world
mailservers have to interact with the real world
Re: FPs on RCVD_ILLEGAL_IP
Posted by Axb <ax...@gmail.com>.
On 04/20/2015 08:45 PM, Reindl Harald wrote:
>
> Am 20.04.2015 um 20:42 schrieb Kevin A. McGrail:
>> On 4/20/2015 2:23 PM, Dianne Skoll wrote:
>>> On Mon, 20 Apr 2015 14:20:35 -0400
>>> "Kevin A. McGrail" <KM...@PCCC.com> wrote:
>>>
>>>> Are you seeing it on a lot of emails?
>>> Over 25000 today; every single one of them from an "...outlook.com"
>>> server. :(
>>>
>>> Regards,
>>>
>>> Dianne.
>> Weird. Any chance you know one of the senders and can ask them to email
>> kmcgrail@pccc.com and raptorreview15@pccc.com with a test? then you and
>> I can compare tests hit, etc.
>
> seems to not hit every mail from outlook.com but when it hits at all
> most are FP's with BAYES_00 and just *one* spam message here the whole
> month with a score of 14 and so catched by enough other rules
>
> looks like RCVD_ILLEGAL_IP does much more harm than good
the rule is good - send your complaint to Microsoft.
0/8 is not assigned.
If a lot of ppl complain there are more chances of being heard.
If ppl blog, blog about this stupidity. If you know someone in the
press, try to get it published.
Re: FPs on RCVD_ILLEGAL_IP
Posted by Reindl Harald <h....@thelounge.net>.
Am 20.04.2015 um 20:42 schrieb Kevin A. McGrail:
> On 4/20/2015 2:23 PM, Dianne Skoll wrote:
>> On Mon, 20 Apr 2015 14:20:35 -0400
>> "Kevin A. McGrail" <KM...@PCCC.com> wrote:
>>
>>> Are you seeing it on a lot of emails?
>> Over 25000 today; every single one of them from an "...outlook.com"
>> server. :(
>>
>> Regards,
>>
>> Dianne.
> Weird. Any chance you know one of the senders and can ask them to email
> kmcgrail@pccc.com and raptorreview15@pccc.com with a test? then you and
> I can compare tests hit, etc.
seems to not hit every mail from outlook.com but when it hits at all
most are FP's with BAYES_00 and just *one* spam message here the whole
month with a score of 14 and so catched by enough other rules
looks like RCVD_ILLEGAL_IP does much more harm than good
Re: FPs on RCVD_ILLEGAL_IP
Posted by Dianne Skoll <df...@roaringpenguin.com>.
On Mon, 20 Apr 2015 14:42:35 -0400
"Kevin A. McGrail" <KM...@PCCC.com> wrote:
> Weird. Any chance you know one of the senders and can ask them to
> email kmcgrail@pccc.com and raptorreview15@pccc.com with a test? then
> you and I can compare tests hit, etc.
Hmm... that'd be awkward because it's not my mail; it's that of some
of our customers.
On Mon, 20 Apr 2015 20:51:53 +0200
Axb <ax...@gmail.com> wrote:
> the rule is good - send your complaint to Microsoft.
> 0/8 is not assigned.
> If a lot of ppl complain there are more chances of being heard.
> If ppl blog, blog about this stupidity. If you know someone in the
> press, try to get it published.
Yes, absolutely. Microsoft needs to be scolded severely for this level of
stupidity.
Regards,
Dianne.
Re: FPs on RCVD_ILLEGAL_IP
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 4/20/2015 2:23 PM, Dianne Skoll wrote:
> On Mon, 20 Apr 2015 14:20:35 -0400
> "Kevin A. McGrail" <KM...@PCCC.com> wrote:
>
>> Are you seeing it on a lot of emails?
> Over 25000 today; every single one of them from an "...outlook.com" server. :(
>
> Regards,
>
> Dianne.
Weird. Any chance you know one of the senders and can ask them to email
kmcgrail@pccc.com and raptorreview15@pccc.com with a test? then you and
I can compare tests hit, etc.
Re: FPs on RCVD_ILLEGAL_IP
Posted by Dianne Skoll <df...@roaringpenguin.com>.
On Mon, 20 Apr 2015 14:20:35 -0400
"Kevin A. McGrail" <KM...@PCCC.com> wrote:
> Are you seeing it on a lot of emails?
Over 25000 today; every single one of them from an "...outlook.com" server. :(
Regards,
Dianne.
Re: FPs on RCVD_ILLEGAL_IP
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 4/20/2015 2:04 PM, Dianne Skoll wrote:
> Not sure if this is still an issue in 3.4, but I'm seeing tons of
> FPs on RCVD_ILLEGAL_IP. Why? Because Microsoft (damn it to hell)
> has started using RESERVED IP ranges internally! Have a look:
>
> Received: from BLUPR10MB0835.namprd10.prod.outlook.com (0.163.216.13)
> by BLUPR10MB0835.namprd10.prod.outlook.com (0.163.216.13)
> with Microsoft SMTP Server (TLS) id 15.1.136.25;
> Mon, 20 Apr 2015 17:43:48 +0000
>
> Is anyone else seeing a sudden uptick in RCVD_ILLEGAL_IP FPs?
I'm using 3.4.1rc2 and before that rc1 and 3.4.1 (trunk). In 45 days of
my personal hand sorted ham corpora I have zero hits on RCVD_ILLEGAL_IP.
Are you seeing it on a lot of emails?
Regards,
KAM