You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Giovanni Bechis <gi...@paclan.it> on 2018/09/18 16:15:48 UTC
Hints needed for spf rule
Hi,
I noticed that Google servers started blocking emails with "suspicious spf records" like for example:
"v=spf1 include:musvc.com include:turbo-smtp.com mx a +all".
Any idea on how to write a rule to catch something like that ?
Thanks & Cheers
Giovanni
Re: Hints needed for spf rule
Posted by Adam Katz <an...@khopis.com>.
On Sep 28, 2018, at 9:48 AM, bOnK wrote:
A better idea might be testing if SPF for a external domain would pass on your own server.
>
> This is what milter greylist does.
> http://hcpnet.free.fr/milter-greylist/
>
> Though probably exceptional, according to the RFC +all *can be* restrictive...
> https://tools.ietf.org/html/rfc7208
>
> A.4. Multiple Requirements Example
>
> Say that your sender policy requires both that the IP address is
> within a certain range and that the reverse DNS for the IP matches.
> This can be done several ways, including the following:
>
> example.com. SPF ( "v=spf1 "
> "-include:ip4._spf.%{d} "
> "-include:ptr._spf.%{d} "
> "+all" )
> ip4._spf.example.com. SPF "v=spf1 -ip4:192.0.2.0/24 +all"
> ptr._spf.example.com. SPF "v=spf1 -ptr +all"
>
> This example shows how the "-include" mechanism can be useful, how an
> SPF record that ends in "+all" can be very restrictive, and the use
> of De Morgan's Law.
>
> --
> b.
Re: Hints needed for spf rule
Posted by RW <rw...@googlemail.com>.
On Thu, 4 Oct 2018 01:29:11 -0400
Adam Katz wrote:
> The ptr mechanism in SPF is officially “do not use” right in the spec
> <https://tools.ietf.org/html/rfc7208#section-5.5>; PTR records aren’t
> vetted (any network operator can assign literally any rDNS to their
> IPs), so it trivializes forgery that would elicit an SPF pass.
It marked as "do not use" because it slow, and it's more sensitive to
packet loss, not because it can be forged. The implementation is
required to check that the DNS is full-circle by performing A-record
look-ups on the rDNS result(s).
Re: Hints needed for spf rule
Posted by Adam Katz <an...@khopis.com>.
(Please ignore my last message. My phone hit “send” randomly.)
On Sep 28, 2018, at 9:48 AM EDT, bOnK wrote:
> A better idea might be testing if SPF for a external domain would pass on your own server.
> This is what milter greylist does.
> http://hcpnet.free.fr/milter-greylist/
That’s interesting! We’d definitely need to ensure external relays for such a rule in SA, though of course this’d also require some plugin dev work. Does anybody have stats on that?
> Though probably exceptional, according to the RFC +all *can be* restrictive...
> https://tools.ietf.org/html/rfc7208#appendix-A.4
>
>> A.4. Multiple Requirements Example
>>
>> Say that your sender policy requires both that the IP address is
>> within a certain range and that the reverse DNS for the IP matches.
>> This can be done several ways, including the following:
>>
>> example.com. SPF ( "v=spf1 "
>> "-include:ip4._spf.%{d} "
>> "-include:ptr._spf.%{d} "
>> "+all" )
>> ip4._spf.example.com. SPF "v=spf1 -ip4:192.0.2.0/24 +all"
>> ptr._spf.example.com. SPF "v=spf1 -ptr +all"
>>
>> This example shows how the "-include" mechanism can be useful, how an
>> SPF record that ends in "+all" can be very restrictive, and the use
>> of De Morgan's Law.
This is amazing. And disgusting.
And the only remotely legitimate usage of either the ptr mechanism or (separately) inanity like invoking De Morgan’s Law, and therefore also +all.
The ptr mechanism in SPF is officially “do not use” right in the spec <https://tools.ietf.org/html/rfc7208#section-5.5>; PTR records aren’t vetted (any network operator can assign literally any rDNS to their IPs), so it trivializes forgery that would elicit an SPF pass.
Using De Morgan to intersect ptr with an un-forgeable requirement alleviates the issues of ptr but it’s much less complicated to merely bless each one in the SPF record.
Any (non-spammer) senders large enough to have issues fitting individual IPs in the max size of a record should definitely not delegate control of SPF to rDNS. They should instead better allocate their IP space for proper control by CIDR or else give up and use an Email Service Provider that actually knows what it’s doing 😉
Re: Hints needed for spf rule
Posted by bOnK <li...@nospam.webmeneer.net>.
On 24-9-2018 17:13, Adam Katz wrote:
> These SPF records are all effectively equivalent (the fourth is Sender
> ID <https://en.wikipedia.org/wiki/Sender_ID>, we'll get to #5 later):
> v=spf1 +all
> v=spf1 all
> v=spf1 all 192.0.2.0/24
> v=spf2.0/mfrom +all
> v=spf1 1.2.3.0/1 128.4.5.0/2 192.6.7.8/3 -all
>
> So therefore I propose regexps like |/^v=spf[12].*[\s+]all\b/| and
> |/^v=spf[12].*\s\?all\b/| (the latter should be very rare and a better
> indication of a clueless admin than a spammer).
>
> The fifth item above permits 0.0.0.0 to 223.255.255.255 and therefore
> only multicast and the reserved Class E network are banned. To address
> this, consider |/^v=spf[12].*[0-9]\/[0-7]\b/|. I haven't observed this
> sort of workaround (yet), but it's the attackers' logical escalation
> in response to this. (I conservatively chose a max mask of /7, though
> I don't think there's any legitimate use of /8, even by the remaining
> Class A holders
> <https://en.wikipedia.org/wiki/List_of_assigned_/8_IPv4_address_blocks#List_of_assigned_/8_blocks>
> like AT&T, HP, and the US DoD—nobody /should/ have an email network
> even approaching a /16 let alone a /8, though note that Google
> currently includes three /16s. I'm not sure where to draw a similar
> "too large" threshold for IPv6; perhaps /32?)
>
> -Adam (still here, sometimes)
>
A better idea might be testing if SPF for a external domain would pass
on your own server.
This is what milter greylist does.
http://hcpnet.free.fr/milter-greylist/
Though probably exceptional, according to the RFC +all *can be*
restrictive...
https://tools.ietf.org/html/rfc7208
A.4. Multiple Requirements Example
Say that your sender policy requires both that the IP address is
within a certain range and that the reverse DNS for the IP matches.
This can be done several ways, including the following:
example.com. SPF ( "v=spf1 "
"-include:ip4._spf.%{d} "
"-include:ptr._spf.%{d} "
"+all" )
ip4._spf.example.com. SPF "v=spf1 -ip4:192.0.2.0/24 +all"
ptr._spf.example.com. SPF "v=spf1 -ptr +all"
This example shows how the "-include" mechanism can be useful, how an
SPF record that ends in "+all" can be very restrictive, and the use
of De Morgan's Law.
--
b.
Re: Hints needed for spf rule
Posted by Adam Katz <an...@khopis.com>.
On 2018-09-22 10:33 am, Kevin A. McGrail wrote:
> On 9/22/2018 10:29 AM, Matus UHLAR - fantomas wrote:
>
>> remove those ?'s: /^v=spf1 .*?all/ and /^v=spf1 .*+all/
>
> Updated. I was trying to stop a greedy regex if someone was doing a
> weird spf but I am overthinking.
These SPF records are all effectively equivalent (the fourth is Sender
ID [1], we'll get to #5 later):
v=spf1 +all
v=spf1 all
v=spf1 all 192.0.2.0/24
v=spf2.0/mfrom +all
v=spf1 1.2.3.0/1 128.4.5.0/2 192.6.7.8/3 -all
So therefore I propose regexps like /^v=spf[12].*[s+]allb/ and
/^v=spf[12].*s?allb/ (the latter should be very rare and a better
indication of a clueless admin than a spammer).
The fifth item above permits 0.0.0.0 to 223.255.255.255 and therefore
only multicast and the reserved Class E network are banned. To address
this, consider /^v=spf[12].*[0-9]/[0-7]b/. I haven't observed this sort
of workaround (yet), but it's the attackers' logical escalation in
response to this. (I conservatively chose a max mask of /7, though I
don't think there's any legitimate use of /8, even by the remaining
Class A holders [2] like AT&T, HP, and the US DoD--nobody _should_ have
an email network even approaching a /16 let alone a /8, though note that
Google currently includes three /16s. I'm not sure where to draw a
similar "too large" threshold for IPv6; perhaps /32?)
-Adam (still here, sometimes)
Links:
------
[1] https://en.wikipedia.org/wiki/Sender_ID
[2]
https://en.wikipedia.org/wiki/List_of_assigned_/8_IPv4_address_blocks#List_of_assigned_/8_blocks
Re: Hints needed for spf rule
Posted by "Kevin A. McGrail" <km...@apache.org>.
On 9/22/2018 10:29 AM, Matus UHLAR - fantomas wrote:
>
> remove those ?'s:
>
> /^v=spf1 .*\?all/
> and
> /^v=spf1 .*\+all/
Updated. I was trying to stop a greedy regex if someone was doing a
weird spf but I am overthinking.
--
Kevin A. McGrail
VP Fundraising, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171
Re: Hints needed for spf rule
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
>On 9/22/2018 9:55 AM, RW wrote:
>> /^v=spf1 .+(\?|\+)all$/
I believe [?+] would do the same easy to read, parse and maybe even to
process (I have no idea how perl RE optimizer works)
>> .+ should be .* or it wont match
>>
>> 'v=spf1 +all'
>>
>> I would remove the '$' as it doesn't appear do anything useful and could
>> prevent matches on weird spf records.
>>
>> It may be worth splitting them into two rules for '?' and '+', there's
>> no dns overhead and they seem like significantly different cases.
hypotetically - masschecks should prove that.
On 22.09.18 09:57, Kevin A. McGrail wrote:
># SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY -
>ifplugin Mail::SpamAssassin::Plugin::AskDNS
> askdns JMQ_SPF_NEUTRAL _SENDERDOMAIN_ TXT /^v=spf1 .*?\?all/
> describe JMQ_SPF_NEUTRAL SPF set to ?all
> score JMQ_SPF_NEUTRAL 0.5
>
> askdns JMQ_SPF_ALL _SENDERDOMAIN_ TXT /^v=spf1 .*?\+all/
> describe JMQ_SPF_ALL SPF set to +all!
> score JMQ_SPF_ALL 0.5
>endif
remove those ?'s:
/^v=spf1 .*\?all/
and
/^v=spf1 .*\+all/
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Windows found: (R)emove, (E)rase, (D)elete
Re: Hints needed for spf rule
Posted by "Kevin A. McGrail" <km...@apache.org>.
On 9/22/2018 9:55 AM, RW wrote:
> On Sat, 22 Sep 2018 08:48:43 -0400
> Kevin A. McGrail wrote:
>
>> On 9/19/2018 6:00 AM, Matus UHLAR - fantomas wrote:
>>> # SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY
>>>> ifplugin Mail::SpamAssassin::Plugin::AskDNS
>>>> askdns JMQ_SPF_NEUTRAL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\?all$/
>>>> describe JMQ_SPF_NEUTRAL_ALL SPF set to ?all!
>>>> score JMQ_SPF_NEUTRAL_ALL 0.5
>>>> endif
>>> do you not check for "+all" by a reason?
>> Would you believe I thought we were? I misread the regex like a year
>> ago. Updating now.
> /^v=spf1 .+(\?|\+)all$/
>
> .+ should be .* or it wont match
>
> 'v=spf1 +all'
>
> I would remove the '$' as it doesn't appear do anything useful and could
> prevent matches on weird spf records.
>
> It may be worth splitting them into two rules for '?' and '+', there's
> no dns overhead and they seem like significantly different cases.
>
# SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY -
ifplugin Mail::SpamAssassin::Plugin::AskDNS
askdns JMQ_SPF_NEUTRAL _SENDERDOMAIN_ TXT /^v=spf1 .*?\?all/
describe JMQ_SPF_NEUTRAL SPF set to ?all
score JMQ_SPF_NEUTRAL 0.5
askdns JMQ_SPF_ALL _SENDERDOMAIN_ TXT /^v=spf1 .*?\+all/
describe JMQ_SPF_ALL SPF set to +all!
score JMQ_SPF_ALL 0.5
endif
Better?
--
Kevin A. McGrail
VP Fundraising, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171
Re: Hints needed for spf rule
Posted by RW <rw...@googlemail.com>.
On Sat, 22 Sep 2018 08:48:43 -0400
Kevin A. McGrail wrote:
> On 9/19/2018 6:00 AM, Matus UHLAR - fantomas wrote:
> > # SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY
> >> ifplugin Mail::SpamAssassin::Plugin::AskDNS
> >> askdns JMQ_SPF_NEUTRAL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\?all$/
> >> describe JMQ_SPF_NEUTRAL_ALL SPF set to ?all!
> >> score JMQ_SPF_NEUTRAL_ALL 0.5
> >> endif
> >
> > do you not check for "+all" by a reason?
>
> Would you believe I thought we were? I misread the regex like a year
> ago. Updating now.
/^v=spf1 .+(\?|\+)all$/
.+ should be .* or it wont match
'v=spf1 +all'
I would remove the '$' as it doesn't appear do anything useful and could
prevent matches on weird spf records.
It may be worth splitting them into two rules for '?' and '+', there's
no dns overhead and they seem like significantly different cases.
Re: Hints needed for spf rule
Posted by "Kevin A. McGrail" <km...@apache.org>.
On 9/19/2018 6:00 AM, Matus UHLAR - fantomas wrote:
> # SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY
>> ifplugin Mail::SpamAssassin::Plugin::AskDNS
>> askdns JMQ_SPF_NEUTRAL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\?all$/
>> describe JMQ_SPF_NEUTRAL_ALL SPF set to ?all!
>> score JMQ_SPF_NEUTRAL_ALL 0.5
>> endif
>
> do you not check for "+all" by a reason?
Would you believe I thought we were? I misread the regex like a year
ago. Updating now.
Re: Hints needed for spf rule
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
>On Tue, Sep 18, 2018 at 12:16 PM Giovanni Bechis <gi...@paclan.it> wrote:
>> I noticed that Google servers started blocking emails with "suspicious spf
>> records" like for example:
>> "v=spf1 include:musvc.com include:turbo-smtp.com mx a +all".
>>
>> Any idea on how to write a rule to catch something like that ?
On 18.09.18 13:01, Kevin A. McGrail wrote:
>It's in KAM.cf, I believe:
>
># SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY
>ifplugin Mail::SpamAssassin::Plugin::AskDNS
> askdns JMQ_SPF_NEUTRAL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\?all$/
> describe JMQ_SPF_NEUTRAL_ALL SPF set to ?all!
> score JMQ_SPF_NEUTRAL_ALL 0.5
>endif
do you not check for "+all" by a reason?
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Posli tento mail 100 svojim znamim - nech vidia aky si idiot
Send this email to 100 your friends - let them see what an idiot you are
Re: Hints needed for spf rule
Posted by Giovanni Bechis <gi...@paclan.it>.
I forgot about KAM.cf, just grepped in rulesrc and found nothing.
Giovanni
On 09/18/18 19:01, Kevin A. McGrail wrote:
> It's in KAM.cf, I believe:
>
> # SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY
> ifplugin Mail::SpamAssassin::Plugin::AskDNS
> askdns JMQ_SPF_NEUTRAL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\?all$/
> describe JMQ_SPF_NEUTRAL_ALL SPF set to ?all!
> score JMQ_SPF_NEUTRAL_ALL 0.5
> endif
>
> --
> Kevin A. McGrail
> VP Fundraising, Apache Software Foundation
> Chair Emeritus Apache SpamAssassin Project
> https://www.linkedin.com/in/kmcgrail - 703.798.0171
>
>
> On Tue, Sep 18, 2018 at 12:16 PM Giovanni Bechis <giovanni@paclan.it <ma...@paclan.it>> wrote:
>
> Hi,
> I noticed that Google servers started blocking emails with "suspicious spf records" like for example:
> "v=spf1 include:musvc.com <http://musvc.com> include:turbo-smtp.com <http://turbo-smtp.com> mx a +all".
>
> Any idea on how to write a rule to catch something like that ?
>
> Thanks & Cheers
> Giovanni
>
Re: Hints needed for spf rule
Posted by "Kevin A. McGrail" <km...@apache.org>.
It's in KAM.cf, I believe:
# SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY
ifplugin Mail::SpamAssassin::Plugin::AskDNS
askdns JMQ_SPF_NEUTRAL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\?all$/
describe JMQ_SPF_NEUTRAL_ALL SPF set to ?all!
score JMQ_SPF_NEUTRAL_ALL 0.5
endif
--
Kevin A. McGrail
VP Fundraising, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171
On Tue, Sep 18, 2018 at 12:16 PM Giovanni Bechis <gi...@paclan.it> wrote:
> Hi,
> I noticed that Google servers started blocking emails with "suspicious spf
> records" like for example:
> "v=spf1 include:musvc.com include:turbo-smtp.com mx a +all".
>
> Any idea on how to write a rule to catch something like that ?
>
> Thanks & Cheers
> Giovanni
>