You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by J Doe <ge...@nativemethods.com> on 2023/07/25 23:20:35 UTC

Ensuring SPF/DKIM for @gmail.com

Hi,

I am currently using SpamAssassin 4.0.0 and I had a question on how I 
can ensure that any e-mail from @gmail.com has a valid SPF and DKIM 
signature.

I am aware that the following can be easily fooled, because it is not 
checking SPF and DKIM:

     welcomelist_from *@gmail.com

... so to ensure valid SPF and DKIM, I believe I would need:

     welcomelist_from_spf  *@gmail.com
     welcomelist_from_dkim *@gmail.com

... or *two* entries.

Is that correct ?

Thanks,

- J

Re: Ensuring SPF/DKIM for @gmail.com

Posted by Grant Taylor via users <us...@spamassassin.apache.org>.
On 7/26/23 2:09 PM, Matija Nalis wrote:
> Only way to make SPF never incorrectly fail/softwail is to use "+all", 
> but that kind of kills its point :-)

I question the veracity of that.

Is SPF failing to perform it's intended function if an unauthorized 
server is blocked from sending email with an envelope from a domain with 
an SPF record ending in "-all"?

I'll defend that SPF is doing exactly what it was designed to do.

I'll agree that some people may be surprised at the message being 
blocked.  But that's a failure of the users understanding of SPF, and 
not a failure of SPF itself.

It may be a bad configuration / SPF record, but again, that's not a 
failure of SPF to do what it was designed to do.

> (actually, even with +all, some sites will fail it - especially
> because of it, as +all is sign of either intentional sloppy spammer
> or incompetent postmaster, both likely leading to spam coming from
> that site).

That is a receiving site applying local policy to override what the 
purported sending site has published.  --  I don't care how many 
receiving sites apply that mentality.  It is still contrary to what the 
purported sending site published.

> Any SPF, no matter how correctly configured, will lead to false
> positives in some cases (e.g. encoutering mailing list or .forward
> not using VERP/SRS).

Is that /truly/ a false positive?  Or did SPF function the way that it 
was designed and configured?  I believe that it did.

I recently helped someone set up SPF for a new domain.  It was working 
fine for a few weeks until they tried to use a 3rd party service to send 
email on their behalf.  I said that's new information and we need to 
update SPF to authorize the 3rd party to send email on their behalf.

Did SPF fail?  Nope.  SPF did exactly what it was designed to do.  The 
user changed what they wanted and needed to update their SPF record 
accordingly.

SPF didn't fail.  The user failed to make a change ahead of time.

> It is inherit in the SPF protocol (which is why
> DMARC checks both DKIM and SPF, in order to reduce, but not
> eliminate, false positives).

DMARC is applying local policy and deciding to ignore the information 
that SPF successfully provided.

Providing unfavorable information is not a failure to provide information.

> We are NOT living in ideal world where everybody implements every
> existing standard.

Agreed.

But we can strive for the ideal world.  By doing so we will end up 
closer to the ideal world than where we started, even if we fall short 
thereof.

> Thus, even most correctly configured SPF will
> sometimes softfail/fail, when it should not.

I question the veracity of "should not".



Grant. . . .

RE: Ensuring SPF/DKIM for @gmail.com

Posted by Marc <Ma...@f1-outsourcing.eu>.
> > >
> > > What does "correctly setup SPF" mean to you?
> >
> > so your ip does not generate a softfail or fail
> 
> Only way to make SPF never incorrectly fail/softwail is to use "+all",
> but that kind of kills its point :-)

+all is in pass
https://datatracker.ietf.org/doc/html/rfc4408#page-8

> (actually, even with +all, some sites will fail it - especially
> because of it, as +all is sign of either intentional sloppy spammer
> or incompetent postmaster, both likely leading to spam coming from
> that site).

I am not even sure if I am able to differentiate on this level in my milter.

> > > What makes your opinion better than someone else's opinion that
> differs?
> > >   (I take it for granted that someone will have a differing
> opinion.)
> >
> > When you configure your spf your result is either pass, softfail or
> fail
> > I think we can agree that a correctly configured spf results in a
> pass, don't you?
> 
> Well *I* don't. Sometimes, maybe even often, it does. But not always.
>
 > Any SPF, no matter how correctly configured, will lead to false
> positives in some cases (e.g. encoutering mailing list or .forward

No not, the sender chooses this setup, so there are no false positives. The sender does not want your server to send email from their domain.
The only reason I can think of, for allowing fail/softfail is if you do not know your own infrastructure wel enough.

.forward should be set to forward with your own email address if spf is configured for external, or if it stays internal, spf should be skipped.

> We are NOT living in ideal world where everybody implements every
> existing standard. Thus, even most correctly configured SPF will
> sometimes softfail/fail, when it should not.
> 

This is just crap. I think 99% of the implemented spf checks are not following your reasoning. It is like you are telling your bank please I would like to use these 2 payment cards to spend from my bank account. 
And because it is not an ideal world, your bank will allow me to spend from your account?





Re: Ensuring SPF/DKIM for @gmail.com

Posted by Benny Pedersen <me...@junc.eu>.
Marc skrev den 2023-07-27 09:48:

>> The oldest mail server log I can find is from mx-in-08 sadly even that
>> one is only from 2005 but confirms we were using it then, quite a bit
>> longer than 2014 :P
> Why retire? To go fishing or so? I think GDPR even prohibits keeping
> very old log files, if there is no specific reason for that.

now that ipv6 exists and working we should all stop using ipv4 problems 
:=)


Re: Ensuring SPF/DKIM for @gmail.com

Posted by Noel Butler <no...@ausics.net>.
On 27/07/2023 17:48, Marc wrote:

>> The oldest mail server log I can find is from mx-in-08 sadly even that
>> one is only from 2005 but confirms we were using it then, quite a bit
>> longer than 2014 :P
> 
> Why retire? To go fishing or so? I think GDPR even prohibits keeping 
> very old log files, if there is no specific reason for that.

Nah, I could never catch anything more than a cold, but I do like 
camping though, peaceful bliss, no server fans droning, no phones 
ringing, no sitting in traffic... even better that most remote spots are 
RF noise free for amateur radio activities whilst other half sleeps her 
head off ;)

As for GDPR, it doesn't affect us, we don't provide services outside our 
own country, also our government, like other "five eyes" nations, are 
perverted control freaks and tend to view all citizens as criminals and 
enemies of the state, so they would be pleased at that duration, we have 
a meta-data retention law that says we must keep CDR's and mail and web 
server logs for "minimum" of 2 years, funny though, they dont want us to 
keep usenet logs, because the vile scum of the earth and warez pups have 
never heard of usenet have they LOL

-- 
Regards,
Noel Butler

This Email, including attachments, may contain legally privileged 
information, therefore at all times remains confidential and subject to 
copyright protected under international law. You may not disseminate 
this message without the authors express written authority to do so.   
If you are not the intended recipient, please notify the sender then 
delete all copies of this message including attachments immediately. 
Confidentiality, copyright, and legal privilege are not waived or lost 
by reason of the mistaken delivery of this message.

RE: Ensuring SPF/DKIM for @gmail.com

Posted by Marc <Ma...@f1-outsourcing.eu>.
> 
> The oldest mail server log I can find is from mx-in-08 sadly even that
> one is only from 2005 but confirms we were using it then, quite a bit
> longer than 2014 :P
> 

Why retire? To go fishing or so? I think GDPR even prohibits keeping very old log files, if there is no specific reason for that.

Re: Ensuring SPF/DKIM for @gmail.com

Posted by Noel Butler <no...@ausics.net>.
On 27/07/2023 13:43, Bill Cole wrote:

> No, SPF pre dates that, 1998 or there abouts if my ageing memory serves 
> me
> It's failing... :)
> 
> SPF originated with an idea of Gordon Fecyk, first written up AFTER he 
> left MAPS in 2001. First ID calling it SPF would have been 2003 or so.

A brief refresher from https://dmarcian.com/history-of-spf/

I'm remembering tiny bits here and there, pfft I'm nearing retirement, 
so maybe that should be sooner rather than latter :)

But 20 years is a long time either way, and the base of my comments 
stand, if you're using an OS or daemon that's that old (or even 10 years 
ago), you STILL have much bigger problems than someone rejecting you on 
SPF :)

The oldest mail server log I can find is from mx-in-08 sadly even that 
one is only from 2005 but confirms we were using it then, quite a bit 
longer than 2014 :P

-- 
Regards,
Noel Butler

This Email, including attachments, may contain legally privileged 
information, therefore at all times remains confidential and subject to 
copyright protected under international law. You may not disseminate 
this message without the authors express written authority to do so.   
If you are not the intended recipient, please notify the sender then 
delete all copies of this message including attachments immediately. 
Confidentiality, copyright, and legal privilege are not waived or lost 
by reason of the mistaken delivery of this message.

Re: Ensuring SPF/DKIM for @gmail.com

Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 2023-07-26 at 23:01:11 UTC-0400 (Thu, 27 Jul 2023 13:01:11 +1000)
Noel Butler <no...@ausics.net>
is rumored to have said:

> On 27/07/2023 10:20, Matija Nalis wrote:
[...]
>> Also, 1990s? Weren't first SPF-alike ideas drafted first time in
>> early-mid 2000s, and SPF itself not published as *proposed* IETF
>> standard until 2014?
>> That was less than a decade ago, barely yesterday :)
>
> No, SPF pre dates that, 1998 or there abouts if my ageing memory 
> serves me

It's failing... :)

SPF originated with an idea of Gordon Fecyk, first written up AFTER he 
left MAPS in 2001. First ID calling it SPF would have been 2003 or so.

Paul Vixie and I had both posted about how a 'reverse MX' or 'Mail 
Sender' record in DNS might work prior to that (and Paul credited 
someone else with the original idea) but those were not fully-developed 
mechanisms that anyone could actually deploy.

-- 
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

Re: Ensuring SPF/DKIM for @gmail.com

Posted by Noel Butler <no...@ausics.net>.
On 27/07/2023 10:20, Matija Nalis wrote:

>> mailing lists have been smart enough for over 20 years to rewrite 
>> sender and
>> not appear as a basic forwarder - which are you are correct, however 
>> there
>> are forwarding abilities to rewrite sender which avoids this, its been 
>> 15
>> years or more since I've used procmail which by default did not.
> 
> I personally know several people who still use procmail today, sooo...
> Your assumption seems to be that EVERYBODY upgrades on regular
> (yearly-or-so?) cycles, and updates their configs to latest recommended
> practices at the same time.

This is ideal but reality is far different, that said, most would not be 
using anything from 1990's, if they are, they are have far bigger issues 
than SPF.

> That at least I can attest is not always the case (I still see
> systems with custom sendmail.cf which nobody dares to touch,
> and with a good reason!)

As above.

> But I won't agree that "it does not exist", nor would I agree that it
> doesn't matter (if it didn't matter to them, people wouldn't be
> asking me to troubleshoot it, and yet they do)

It "does, not matter", you can't help those who wont help themselves, 
I'm sure we all remember this back in days when banks and governments 
wouldt run compliant DNS, they all expected us to whitelist them, when 
they realised that was not going to happen en masse, they got their act 
together and fixed their stuff, now, at least in this country, they woke 
up and realised the benefits so much so, the govt here is a strong 
proponent of DMARC and mandates all federal govt depts to use it (though 
I've discovered some that dont)

> Good for you. But that is anecdotal - you are certainly not 
> participating
> in every mailing list in existence,

I'm on 117 mailing lists - not that I have time these days to read much 
of it, family life is more important, in past couple weeks I just found 
a few hours to peruse some :)

> So, still in 2023, I have to deal with SPF (and DKIM) failing due to 
> such forwarders/ML (as well
> as misconfigurations, of course)

DKIM is a total failure with mailing lists, but DKIM - unlike SPF in a 
typical setup, is not an out-right reject at MTA level.

> Also, 1990s? Weren't first SPF-alike ideas drafted first time in
> early-mid 2000s, and SPF itself not published as *proposed* IETF
> standard until 2014?
> That was less than a decade ago, barely yesterday :)

No, SPF pre dates that, 1998 or there abouts if my ageing memory serves 
me correct, 2014 might have been the SPF RR type, which certain cretins 
from the debian world fought long hard against as their dist versions of 
bind didnt understand it it was that old (heaven forbid debian users ran 
modern software - I hope thats changed since but somehow I suspect 
not...)

-- 
Regards,
Noel Butler

This Email, including attachments, may contain legally privileged 
information, therefore at all times remains confidential and subject to 
copyright protected under international law. You may not disseminate 
this message without the authors express written authority to do so.   
If you are not the intended recipient, please notify the sender then 
delete all copies of this message including attachments immediately. 
Confidentiality, copyright, and legal privilege are not waived or lost 
by reason of the mistaken delivery of this message.

Re: Ensuring SPF/DKIM for @gmail.com

Posted by Grant Taylor via users <us...@spamassassin.apache.org>.
On 7/26/23 7:20 PM, Matija Nalis wrote:
> I'd appreciate more civil expressions of disagreement

+1

> I personally know several people who still use procmail today, sooo...

+1

> That at least I can attest is not always the case (I still see 
> systems with custom sendmail.cf which nobody dares to touch, 
> and with a good reason!)

Is said sendmail.cf based on a sendmail.mc file or is it older / bespoke?

> Yeah, I agree that it sure would be nice if world worked that way and 
> everybody upgraded regularly and configured them according to latest 
> BCPs, but around here at least, it sometimes (I'm avoiding to say 
> "often") doesn't.

I agree with following the latest BCPs *if* *reasonably* *possible* *to* 
*do* *so*.

But I don't agree with upgrading just because something is old, 
especially if it's still working.

After all, Ethernet is 50 (?) years old and TCP/IP is no spring chicken. 
  Yet we are still using them.

> There are quite a few systems that someone knowledgable setup some 
> time back, and after they've gone to greener pastures, nobody have 
> touched them, yet they continue to use them.

Sometimes you find the diamond in the rough that knows how to care and 
feed said systems and does tweak them to abide by / support BCPs.

> Sure, I'll be first to agree that it is bad and should be fixed.

What is the actual problem that /needs/ /to/ /be/ /fixed/?

Procmail can forward emails without re-writing the envelope if the MTA 
does the envelope re-writing for it.

Just because something is old and doesn't provide the latest and 
greatest feature doesn't mean that something else can't assist and 
provide said feature.

The complete solution needs to provide the features.  Not all features 
need to be provided by specific components.

Telnet over a VPN is perfectly fine especially if the destination 
doesn't support something better.

> But I won't agree that "it does not exist", nor would I agree that it 
> doesn't matter (if it didn't matter to them, people wouldn't be 
> asking me to troubleshoot it, and yet they do)

#truth +1

> Good for you. But that is anecdotal - you are certainly not participating 
> in every mailing list in existence, nor do you contact all people on the 
> planet which use every kind of mail forwarder.
> 
> Neither do I, but I service lots of systems of other people that do, 
> and with many people, the chances rise. So, still in 2023, I have to 
> deal with SPF (and DKIM) failing due to such forwarders/ML (as well 
> as misconfigurations, of course)

+1

> Also, 1990s? Weren't first SPF-alike ideas drafted first time in 
> early-mid 2000s, and SPF itself not published as *proposed* IETF 
> standard until 2014?

I was wondering, but I couldn't be bothered to look up the dates.  I 
think I started using SPF in the mid-2000s.  Maybe late 2000s.

> That was less than a decade ago, barely yesterday :)

LOL



Grant. . . .

Re: Ensuring SPF/DKIM for @gmail.com

Posted by Matija Nalis <mn...@voyager.hr>.
On Thu, Jul 27, 2023 at 07:11:59AM +1000, Noel Butler wrote:
> On 27/07/2023 05:09, Matija Nalis wrote:
> 
> > Any SPF, no matter how correctly configured, will lead to false
> > positives in some cases (e.g. encoutering mailing list
> 
> B.S.

I'd appreciate more civil expressions of disagreement, though, if
this means what I think it means.

> mailing lists have been smart enough for over 20 years to rewrite sender and
> not appear as a basic forwarder - which are you are correct, however there
> are forwarding abilities to rewrite sender which avoids this, its been 15
> years or more since I've used procmail which by default did not.

I personally know several people who still use procmail today, sooo...
Your assumption seems to be that EVERYBODY upgrades on regular
(yearly-or-so?) cycles, and updates their configs to latest recommended 
practices at the same time.

That at least I can attest is not always the case (I still see
systems with custom sendmail.cf which nobody dares to touch, 
and with a good reason!) 

Yeah, I agree that it sure would be nice if world worked that way and
everybody upgraded regularly and configured them according to latest
BCPs, but around here at least, it sometimes (I'm avoiding to say
"often") doesn't.

There are quite a few systems that someone knowledgable setup some
time back, and after they've gone to greener pastures, nobody have
touched them, yet they continue to use them. 

Sure, I'll be first to agree that it is bad and should be fixed. 

But I won't agree that "it does not exist", nor would I agree that it
doesn't matter (if it didn't matter to them, people wouldn't be
asking me to troubleshoot it, and yet they do)

> If you are going to dry-reach to support an argument, please use modern

I'm not aware of that "dry-reach" idiom, would you care to explain?

> facts and not 1990's. I was a *very* early adopter of SPF back in late 90's
> and have had zero issues in 20 years in using SPF (as expected as an early
> adopter, teething issues as with all software needed fine tuning in very
> early days)

Good for you. But that is anecdotal - you are certainly not participating 
in every mailing list in existence, nor do you contact all people on the 
planet which use every kind of mail forwarder.

Neither do I, but I service lots of systems of other people that do,
and with many people, the chances rise. So, still in 2023, I have to
deal with SPF (and DKIM) failing due to such forwarders/ML (as well
as misconfigurations, of course)

Also, 1990s? Weren't first SPF-alike ideas drafted first time in
early-mid 2000s, and SPF itself not published as *proposed* IETF
standard until 2014? 
That was less than a decade ago, barely yesterday :)

-- 
Opinions above are GNU-copylefted.

Re: Ensuring SPF/DKIM for @gmail.com

Posted by Noel Butler <no...@ausics.net>.
On 27/07/2023 05:09, Matija Nalis wrote:

> Any SPF, no matter how correctly configured, will lead to false
> positives in some cases (e.g. encoutering mailing list

B.S.
mailing lists have been smart enough for over 20 years to rewrite sender 
and not appear as a basic forwarder - which are you are correct, however 
there are forwarding abilities to rewrite sender which avoids this, its 
been 15 years or more since I've used procmail which by default did not.

If you are going to dry-reach to support an argument, please use modern 
facts and not 1990's. I was a *very* early adopter of SPF back in late 
90's and have had zero issues in 20 years in using SPF (as expected as 
an early adopter, teething issues as with all software needed fine 
tuning in very early days)

-- 
Regards,
Noel Butler

This Email, including attachments, may contain legally privileged 
information, therefore at all times remains confidential and subject to 
copyright protected under international law. You may not disseminate 
this message without the authors express written authority to do so.   
If you are not the intended recipient, please notify the sender then 
delete all copies of this message including attachments immediately. 
Confidentiality, copyright, and legal privilege are not waived or lost 
by reason of the mistaken delivery of this message.

Re: Ensuring SPF/DKIM for @gmail.com

Posted by Matija Nalis <mn...@voyager.hr>.
On Wed, Jul 26, 2023 at 06:44:32PM +0000, Marc wrote:
> > At the risk of starting a flame war...
> > 
> > What does "correctly setup SPF" mean to you?
> 
> so your ip does not generate a softfail or fail

Only way to make SPF never incorrectly fail/softwail is to use "+all",
but that kind of kills its point :-)

(actually, even with +all, some sites will fail it - especially
because of it, as +all is sign of either intentional sloppy spammer
or incompetent postmaster, both likely leading to spam coming from
that site).

> > What makes your opinion better than someone else's opinion that differs?
> >   (I take it for granted that someone will have a differing opinion.)
> 
> When you configure your spf your result is either pass, softfail or fail
> I think we can agree that a correctly configured spf results in a pass, don't you?

Well *I* don't. Sometimes, maybe even often, it does. But not always.

Any SPF, no matter how correctly configured, will lead to false
positives in some cases (e.g. encoutering mailing list or .forward
not using VERP/SRS). It is inherit in the SPF protocol (which is why
DMARC checks both DKIM and SPF, in order to reduce, but not
eliminate, false positives).

We are NOT living in ideal world where everybody implements every
existing standard. Thus, even most correctly configured SPF will
sometimes softfail/fail, when it should not. 

Trying to pretend that the world is ideal is not really good idea;
one might as well pretend that spam does not exist and save all that
time wasted on implementing antispam measures :-)

-- 
Opinions above are GNU-copylefted.

Re: Ensuring SPF/DKIM for @gmail.com

Posted by Noel Butler <no...@ausics.net>.
On 27/07/2023 18:11, Marc wrote:

> I am always using -all. I honestly can't think of a good argument to 
> use anything else.

I agree.

It's my belief that ~all is only useful for a "production entry test 
phase", once your happy, move to -all

Like DMARC's p=none it's a "getting it going" method that's for you to 
get shit right, then move to p=quarantine, although from memory some 
European countries (Germany?) require or use to require you to either 
accept the message and deliver it, or outright block it with a reject 
message, I'd like to think they've changed that though.

-- 
Regards,
Noel Butler

This Email, including attachments, may contain legally privileged 
information, therefore at all times remains confidential and subject to 
copyright protected under international law. You may not disseminate 
this message without the authors express written authority to do so.   
If you are not the intended recipient, please notify the sender then 
delete all copies of this message including attachments immediately. 
Confidentiality, copyright, and legal privilege are not waived or lost 
by reason of the mistaken delivery of this message.

RE: Ensuring SPF/DKIM for @gmail.com

Posted by Marc <Ma...@f1-outsourcing.eu>.
> 
> >> I assume that you mean so that your outbound SMTP server is actually
> >> authorized in some capacity and fall under "all".  Is that correct?
> 
> ... and does NOT dall under "all".
> 
> On 27.07.23 08:11, Marc wrote:
> >indeed afaik -all is all authorized
> 
> pardon me? -all means everyone except previously mentioned it
> UNAUTHORIZED
> to send mail.
> 
> fantomas.sk.            43200   IN      TXT     "v=spf1 mx -all"
> 
> meant only "mx" servers for fantomas.sk can send mail from this domain,
> all
> the rest is unauthorized.
> 

so we agree :). "-all" means only authorized can deliver, authorized as in mentioned in the text record.

Re: Ensuring SPF/DKIM for @gmail.com

Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
>> I assume that you mean so that your outbound SMTP server is actually
>> authorized in some capacity and fall under "all".  Is that correct?

... and does NOT dall under "all".

On 27.07.23 08:11, Marc wrote:
>indeed afaik -all is all authorized

pardon me? -all means everyone except previously mentioned it UNAUTHORIZED 
to send mail.

fantomas.sk.            43200   IN      TXT     "v=spf1 mx -all"

meant only "mx" servers for fantomas.sk can send mail from this domain, all 
the rest is unauthorized.

-- 
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...

RE: Ensuring SPF/DKIM for @gmail.com

Posted by Marc <Ma...@f1-outsourcing.eu>.
> 
> I assume that you mean so that your outbound SMTP server is actually
> authorized in some capacity and fall under "all".  Is that correct?

indeed afaik -all is all authorized

> > When you configure your spf your result is either pass, softfail or
> fail
> > I think we can agree that a correctly configured spf results in a
> pass,
> > don't you?
> 
> I agree that known and authorized sources of email for my domain are
> authorized in the SPF record for my domain.
> 
> I thought you were alluding to a particular value of "all".  I did not
> understand your statement to be about servers being authorized or not.

gmail/google/outlook are using ~all, resulting in the softfail. For those networks if have decided to treat only their ~all as a -all. 

> I've had a number of conversations with people of late that seem to
> dislike "-all" almost as much as others dislike "+all".
> 

I am always using -all. I honestly can't think of a good argument to use anything else. 

Re: Ensuring SPF/DKIM for @gmail.com

Posted by Grant Taylor via users <us...@spamassassin.apache.org>.
On 7/26/23 1:44 PM, Marc wrote:
> so your ip does not generate a softfail or fail

I assume that you mean so that your outbound SMTP server is actually 
authorized in some capacity and fall under "all".  Is that correct?

> When you configure your spf your result is either pass, softfail or fail
> I think we can agree that a correctly configured spf results in a pass,
> don't you?

I agree that known and authorized sources of email for my domain are 
authorized in the SPF record for my domain.

I thought you were alluding to a particular value of "all".  I did not 
understand your statement to be about servers being authorized or not.

I've had a number of conversations with people of late that seem to 
dislike "-all" almost as much as others dislike "+all".



Grant. . . .

RE: Ensuring SPF/DKIM for @gmail.com

Posted by Marc <Ma...@f1-outsourcing.eu>.
> At the risk of starting a flame war...
> 
> What does "correctly setup SPF" mean to you?

so your ip does not generate a softfail or fail

> What makes your opinion better than someone else's opinion that differs?
>   (I take it for granted that someone will have a differing opinion.)

When you configure your spf your result is either pass, softfail or fail
I think we can agree that a correctly configured spf results in a pass, don't you?
 
> I get your server your rules.  But your server / your rules doesn't
> translate to someone else's server any more than someone else's rules
> translate to your server.
> 

I agree

Re: Ensuring SPF/DKIM for @gmail.com

Posted by Grant Taylor via users <us...@spamassassin.apache.org>.
On 7/26/23 1:44 AM, Marc wrote:
> asking them to correctly setup spf is mostly enough.

At the risk of starting a flame war...

What does "correctly setup SPF" mean to you?

What makes your opinion better than someone else's opinion that differs? 
  (I take it for granted that someone will have a differing opinion.)

I get your server your rules.  But your server / your rules doesn't 
translate to someone else's server any more than someone else's rules 
translate to your server.



Grant. . . .

Re: Ensuring SPF/DKIM for @gmail.com

Posted by Noel Butler <no...@ausics.net>.
On 26/07/2023 17:34, Benny Pedersen wrote:

> milters should not be spam scanners, spamassassin is better

SA is perl, perl is faster and better resource nice than python garbage, 
but perl is still slow compared to C, that is why milters will win out 
everytime.

milter-regex is also light and super speedy, it stops a lot of trash 
before postfix even accepts the message to give to SA

Frankly google is just trash anyway, so anything that blocks 
gmail/google spam is a great idea. (have they stopped google groups from 
backscatter yet, probably not, they are too busy fscking over youtube)

-- 
Regards,
Noel Butler

- Ensuring my "long sig" is even longer, just for Benny

This Email, including attachments, may contain legally privileged 
information, therefore at all times remains confidential and subject to 
copyright protected under international law. You may not disseminate 
this message without the authors express written authority to do so.   
If you are not the intended recipient, please notify the sender then 
delete all copies of this message including attachments immediately. 
Confidentiality, copyright, and legal privilege are not waived or lost 
by reason of the mistaken delivery of this message.

This I care not about mailing lists, although this is only a 
list/newsletter account its not my personal account

Its one of my older formal but not personal addresses nbot really used 
in that lght now.

thinking of other stuff to put in to annoy Benny but damn running outof 
time, time to go home its 5.50 pm

Re: Ensuring SPF/DKIM for @gmail.com

Posted by Grant Taylor via users <us...@spamassassin.apache.org>.
On 7/27/23 6:25 AM, Matus UHLAR - fantomas wrote:
> I use spamass-milter on my system and amavisd-milter on other systems 
> especially to be able to reject spam at SMTP time. Definitely a good thing.

:-)

> You just should not use it for "outgoing" mail from your clients, so 
> they don't complain about sending mail taking ages.

Eh ....

I'm an advocate for spam filtering all email coming into the server, 
even email coming into the MSA on it's way out to another server across 
the Big Bad Internet.

> You may need to limit SA processing to some 270 seconds to avoid 
> 5-minutes timeout (which contradicts RFC5321 section 4.5.3.2.6. which 
> mandates 10-minute timeout on DATA termination, but here we are).

This is one of the reasons that I mentioned configuring the MSA with 
much longer timeouts in another recent thread (though I don't remember 
which list).

I have in the past configured an MSA to accept messages (with really 
long for email timeouts) and relay the messages through the local MTA 
where I applied the filtering that I'm talking about.

This allowed clients, even on dial up, to send larger email with 
attachments to not time out /and/ to be able to be filtered to avoid 
contributing to spam if (when) accounts were compromised.

Seeing as how the MUAs authenticated to the MSA to send, I have a very 
good idea who I needed to have a chat with in the event that something 
unsavory was making it through the MSA to the filtering MTA, or worse 
out to the Big Bad Internet.



Grant. . . .

Re: Ensuring SPF/DKIM for @gmail.com

Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
>On 7/26/23 2:34 AM, Benny Pedersen wrote:
>>milters should not be spam scanners, spamassassin is better

On 26.07.23 13:32, Grant Taylor via users wrote:
>{spamass-milter,milter-spamc} combined with SpamAssassin cause me to 
>question the veracity of that statement.

+1

>Milter implies doing the filtering during the SMTP transaction.  I 
>consider the ability to reject messages that SpamAssassin declares as 
>(bad enough) spam at SMTP time to be a good thing.

I use spamass-milter on my system and amavisd-milter on other systems 
especially to be able to reject spam at SMTP time. Definitely a good thing.

You just should not use it for "outgoing" mail from your clients, so they 
don't complain about sending mail taking ages.

You may need to limit SA processing to some 270 seconds to avoid 5-minutes 
timeout (which contradicts RFC5321 section 4.5.3.2.6. which mandates 
10-minute timeout on DATA termination, but here we are).

-- 
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Your mouse has moved. Windows NT will now restart for changes to take
to take effect. [OK]

Re: Ensuring SPF/DKIM for @gmail.com

Posted by Grant Taylor via users <us...@spamassassin.apache.org>.
On 7/26/23 2:34 AM, Benny Pedersen wrote:
> milters should not be spam scanners, spamassassin is better

{spamass-milter,milter-spamc} combined with SpamAssassin cause me to 
question the veracity of that statement.

Milter implies doing the filtering during the SMTP transaction.  I 
consider the ability to reject messages that SpamAssassin declares as 
(bad enough) spam at SMTP time to be a good thing.

> maybe use bind9 rpz to change spf data for stupid domains, mostly 
> freemail domains in this category ? :)

Will you provide a description of what you would have an RPZ do?

I know that you can specify local data overriding the upstream DNS zone 
data.

All of the local data would be locally and artificially baked.  I guess 
you could do some sort of query and create a fresh RPZ record with a 
different "all" value.  Or are you thinking something else?

This might be a use case for RPS to more dynamically alter the record. 
But I've not messed with RPS much.  Conversely I've done a lot with RPZ.



Grant. . . .

Re: Ensuring SPF/DKIM for @gmail.com

Posted by Benny Pedersen <me...@junc.eu>.
Marc skrev den 2023-07-26 08:44:

>> blocklist_from *@gmail.com
>> welcomelist_auth *@gmail.com
>> makes it perfect :)
>> if both dkim and spf is pass, it will get neutral scores

> I found this to be not sufficient (assuming the above pass is ~all).
> gmail has spf ~all.

set softfail score to 100, solved

> So I have made an exception for the google network in milter and
> everything from the gmail / google that would fail an -all spf I
> reject.

milters should not be spam scanners, spamassassin is better

> There is only a few legitimate domains that will be targetted by this,
> but asking them to correctly setup spf is mostly enough.

maybe use bind9 rpz to change spf data for stupid domains, mostly 
freemail domains in this category ? :)


RE: Ensuring SPF/DKIM for @gmail.com

Posted by Marc <Ma...@f1-outsourcing.eu>.
> 
> blocklist_from *@gmail.com
> welcomelist_auth *@gmail.com
> 
> makes it perfect :)
> 
> if both dkim and spf is pass, it will get neutral scores
> 

I found this to be not sufficient (assuming the above pass is ~all). gmail has spf ~all. 

So I have made an exception for the google network in milter and everything from the gmail / google that would fail an -all spf I reject.
There is only a few legitimate domains that will be targetted by this, but asking them to correctly setup spf is mostly enough.

 

Re: Ensuring SPF/DKIM for @gmail.com

Posted by Benny Pedersen <me...@junc.eu>.
J Doe skrev den 2023-07-26 01:52:

> Thanks for your reply - perfect: welcomelist_auth is exactly what I
> was looking for!

blocklist_from *@gmail.com
welcomelist_auth *@gmail.com

makes it perfect :)

if both dkim and spf is pass, it will get neutral scores

but if fail it will get spamscored, thanks for this goes to John Hardin

i just say don't welcomelist all gmail senders



Re: Ensuring SPF/DKIM for @gmail.com

Posted by J Doe <ge...@nativemethods.com>.
On 2023-07-25 19:39, Benny Pedersen wrote:

> J Doe skrev den 2023-07-26 01:20:
>  
> its a one liner with welcomelist_auth

Hi Benny,

Thanks for your reply - perfect: welcomelist_auth is exactly what I was 
looking for!

- J


Re: Ensuring SPF/DKIM for @gmail.com

Posted by Benny Pedersen <me...@junc.eu>.
J Doe skrev den 2023-07-26 01:20:

> I am currently using SpamAssassin 4.0.0 and I had a question on how I
> can ensure that any e-mail from @gmail.com has a valid SPF and DKIM
> signature.

incorrect questions gives incorrect answers

> I am aware that the following can be easily fooled, because it is not
> checking SPF and DKIM:
> 
>     welcomelist_from *@gmail.com

will accept forged senders

> ... so to ensure valid SPF and DKIM, I believe I would need:
> 
>     welcomelist_from_spf  *@gmail.com
>     welcomelist_from_dkim *@gmail.com
> 
> ... or *two* entries.
> 
> Is that correct ?

yes

read perldoc Mail::SpamAssassin::Conf in the section auth

welcomelist_from user@example.com
         Previously whitelist_from which will work interchangeably until 
4.1.

         Used to welcomelist sender addresses which send mail that is 
often
         tagged (incorrectly) as spam.

         Use of this setting is not recommended, since it blindly trusts 
the
         message, which is routinely and easily forged by spammers and 
phish
         senders. The recommended solution is to instead use
         "welcomelist_auth" or other authenticated welcomelisting 
methods, or
         "welcomelist_from_rcvd".

its a one liner with welcomelist_auth


Re: Ensuring SPF/DKIM for @gmail.com

Posted by David B Funk <db...@engineering.uiowa.edu>.
If you do that you will guarantee yourself to get bunches of spam that might 
otherwise be tagged by SA.

the "welcomelist" mechanism says:
  Anybody who matches this criteria we consider strongly not to be spam 
(regardless of how spammy all the other metrics may say it is).

You should "welcomelist" stuff that you want to guarantee passage of, regarless 
of all other considerations.

Given that Google:
a) SPF & DKIMs all the stuff that comes out of their system
b) has lots of spammers who have Gmail accounts and spew spam from them.
c) does not seem to care two hoots about (b) and lets (b) happen even in the
   case of reports.

So if you do those lines (or the more all-encompasing 'welcomelist_auth' form) 
you guarantee those spammers a free ride into your system.

Now if you want to find those critters that forge "name@gmail.com" as a sender
you'll need to create a custom rule set:
1) a non-scoring rule that fires when from == "@gmail.com"
2) a 'meta rule' that says if-from-gmail && not DKIM_VALID then give 
it a spam score

DKIM_SIGNED is a standard SA rule that detects a properly valid DKIM or DK 
signature.


On Tue, 25 Jul 2023, J Doe wrote:

> Hi,
>
> I am currently using SpamAssassin 4.0.0 and I had a question on how I can 
> ensure that any e-mail from @gmail.com has a valid SPF and DKIM signature.
>
> I am aware that the following can be easily fooled, because it is not 
> checking SPF and DKIM:
>
>    welcomelist_from *@gmail.com
>
> ... so to ensure valid SPF and DKIM, I believe I would need:
>
>    welcomelist_from_spf  *@gmail.com
>    welcomelist_from_dkim *@gmail.com
>
> ... or *two* entries.
>
> Is that correct ?
>
> Thanks,
>
> - J
>
>

-- 
Dave Funk                               University of Iowa
<dbfunk (at) engineering.uiowa.edu>     College of Engineering
319/335-5751   FAX: 319/384-0549        1256 Seamans Center, 103 S Capitol St.
Sys_admin/Postmaster/cell_admin         Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{

Re: Ensuring SPF/DKIM for @gmail.com

Posted by Greg Troxel <gd...@lexort.com>.
J Doe <ge...@nativemethods.com> writes:

> I am currently using SpamAssassin 4.0.0 and I had a question on how I
> can ensure that any e-mail from @gmail.com has a valid SPF and DKIM
> signature.

You should phrase what you want more carefully.  What I think you said
is:

  I want that if mail comes in with a From: of *@gmail,com and if either
  SPF or DKIM fails, then I want to reject that mail.

Be careful what you wish for.  That will cause mailinglist mail to be
rejected.  Probably you should accept if DKIM passes, regardless of SPF.
And maybe SPF without DKIM, but I doubt there is much mail like that.

> I am aware that the following can be easily fooled, because it is not
> checking SPF and DKIM:
>
>     welcomelist_from *@gmail.com

Not only that, it says that any such mail is accepted, which is not what
you said.

>
> ... so to ensure valid SPF and DKIM, I believe I would need:
>
>     welcomelist_from_spf  *@gmail.com
>     welcomelist_from_dkim *@gmail.com
>
> ... or *two* entries.

That means that anything that passes spf is accepted and anything that
passes dkim.  But that's not what you said; you said "ensure" which
means that you *reject* things that do not have both.   And then you
stilld do spam filtering on things that you didn't reject outright.

There is a lot of DKIM-signed SPF-compliant spam from gmail.  They let
people sign up for accounts, and some of them spam.  So "accept all mail
from gmail" is not a sensible policy.

Rejecting mail that claims to be grom gmail but isn't is more sensible,
but you need to understand that many mailinglists (incorrectly) munge
mail and cause it to fail DKIM, and of course SPF doesn't match.


What I do is assign a few spam points for gmail and add
welcomelist_from_dkim for people I know, or welcomelist_from_rcvd for
people on lists from the list sender.