You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Ben Lentz <bl...@channing-bete.com> on 2005/09/28 20:31:35 UTC

SPF and Upgrade to SA3.1

Greetings,
Since upgrading from 3.0.4 to 3.1.0, my SPF checks no longer work. It 
would seem that the information being passed Mail::SPF::Query->new does 
not contain the sender's domain, but rather the FQDN of the last system 
that sent the email. This FQDN does not have a TXT record, and so SPF is 
never successful.

Non-partisan example:
[24820] dbg: spf: query for /72.14.204.200/qproxy.gmail.com: result: 
none, comment: SPF: domain of sender qproxy.gmail.com does not designate 
mailers

gmail.com's TXT record contains the correct SPF, but for some reason 
since upgrading to 3.1.0, SA is looking up qproxy.gmail.com's TXT record 
instead, and now everything SPF is broken.

I tried spoofing mail to myself and didn't get any SPF_FAIL tag, nor do 
I still get consistent SPF_HELO_PASS or SPF_PASS tags when sending from 
gmail.

Is my testing of gmail's information faulty? What has changed between 
3.0.4 and 3.1.0 SPF-wise that would cause SPF to stop working?

Thanks for any help you can provide.

Re: SPF and Upgrade to SA 3.1

Posted by Ben Lentz <bl...@channing-bete.com>.
Heh, That was supposed to be a joke; not very funny, I guess.

----- Original Message -----
*From:* Theo Van Dinter <fe...@apache.org>
*Sent:* 09/29/2005 02:57:10 PM
*To:* users@spamassassin.apache.org
*Subject:* SPF and Upgrade to SA 3.1



>On Thu, Sep 29, 2005 at 02:47:05PM -0400, Ben Lentz wrote:
>  
>
>>SPF stuff. Technology vendors everywhere are telling me that if I 
>>implement SPF and DK that the entire plannet will be spam free.
>>    
>>
>
>Just FWIW:
>Those technology vendors are confused.  Neither SPF nor DK are anti-spam
>technologies.  They're anti-forging technologies.  That can potentially
>lead to less spam, but that's not their direct target.
>
>  
>

Re: SPF and Upgrade to SA 3.1

Posted by Theo Van Dinter <fe...@apache.org>.
On Thu, Sep 29, 2005 at 02:47:05PM -0400, Ben Lentz wrote:
> SPF stuff. Technology vendors everywhere are telling me that if I 
> implement SPF and DK that the entire plannet will be spam free.

Just FWIW:
Those technology vendors are confused.  Neither SPF nor DK are anti-spam
technologies.  They're anti-forging technologies.  That can potentially
lead to less spam, but that's not their direct target.

-- 
Randomly Generated Tagline:
"if you don't know what Unix is, get off the internet." - Catherine Skidmore

Re: SPF and Upgrade to SA 3.1

Posted by Ben Lentz <bl...@channing-bete.com>.



> Yes- I would rather have correct results than just results.
>
> Okay, so the problem is with my MTA moving the Return-Path header 
> below the Received headers. It's breaking spamassassin's ability to 
> check perfectly compliant SPF records. I'm using stock versions of 
> sendmail 8.13 on all my boxes, so I guess my only recourse is to go to 
> the sendmail folks and ask them why.
FWIW, a year later: My problem was that final (mailbox) delivery was not 
being made on the MTA that was running SA, so I had no Return-Path 
header at all. I understand that the Return-Path is supposed to be added 
only by the MTA that is doing final delivery. The fix for my MTA, 
sendmail 8.13, was to follow the instructions provided at 
http://wiki.apache.org/spamassassin/EnvelopeSenderInReceived, and to 
modify my sendmail configuration on the sending (MX) MTA, to add the 
envelope-from information in the received headers. For sendmail 
(sendmail.mc):
define(`_REC_BY_', `$.by $j (envelope-from $f) ($v/$Z)$?r with $r$. id 
$i$?{tls_version}')

My other problem, which was solved much earlier with Daryl's advice, was 
that the MTA running SA is not the initial recipient (MX) of an inbound 
email, either. So that's where properly defining trusted_networks for SA 
making sure to include the MX MTA in the address spec, *and* enabling 
always_trust_envelope_sender 1 becomes imperative.

This will make the result of the MAIL FROM: <> available to other MTAs 
in the chain, allowing SpamAssassin to look back at what this data was 
at a previous MTA hop, so long as it is trusted.

Hopefully this information will be useful to anyone else who wishes to 
use the SPF checks in SA on a non-"single-box" MTA configuration. I know 
it wasn't obvious to me, but then again, I'm not the brightest bulb in 
the box. ;-)

Things are working great now, but please let me know if I've messed up 
any of this information.
>
>
> It's probably not worth it- I'm really not in the mood to think about 
> recompliling sendmail at the moment.
>
> Thanks for your help.
>
> ----- Original Message -----
> *From:* "Daryl C. W. O'Shea" <sp...@dostech.ca>
> *Sent:* 09/29/2005 03:36:15 PM
> *To:* Ben Lentz <bl...@channing-bete.com>
> *Cc:* users@spamassassin.apache.org
> *Subject:* SPF and Upgrade to SA 3.1
>
>
>
>> Ben Lentz wrote:
>>
>>> _You_ are _welcome_.
>>>
>>> Get it moved? - Hmmm... Ala-kazamm! - Oh, that didn't work. Okay, so 
>>> magic isn't going to get it moved, and I'm all out of ideas.
>>
>>
>> I can only suggest starting another thread here or "somewhere else 
>> applicable" that asks "this is the software I'm using, why is my 
>> return-path being appended to the bottom and not prepended to the top?".
>>
>> If you're specific in the software you're using I'd be surprised if 
>> someone can't give you a reason why and how to fix it.
>>
>>
>>> I still don't understand why I used to get SPF_HELO_PASSes with 
>>> 3.0.4 and I don't with 3.1. The world hasn't changed, just my SA 
>>> version. I guess SA was doing it "wrong" before, but is doing it 
>>> "right" now? Is that the concrete explanation?
>>
>>
>> YES!  You'll continue to get SPF_HELO_PASSes with SA 3.1.0 too, for 
>> domains that fully implement the draft and publish SPF records for 
>> each host.  SA 3.0 did INCORRECTLY fire SPF_HELO_PASSes for domains 
>> it shouldn't have.  Surely you would prefer correct results and not 
>> just results.
>>
>> The fact of the matter, though, is that nobody really cares about 
>> SPF_HELO_* checks.  Some, usually smaller, domains publish them but 
>> that's about it.
>>
>> Heck, even SA doesn't care about SPF_HELO_PASS:
>>
>> score SPF_HELO_PASS -0.001
>>
>>
>> Also FWIW, I ran your test message through SA 3.0.4-r165054 and got 
>> the incorrect SPF_HELO_PASS result.  I did NOT get an SPF_PASS result 
>> though because:
>>
>> debug: Return-Path header found after 1 or more Received lines, 
>> cannot trust envelope-from
>>
>>
>>> The idea I'm obsessed with is that I moved to SA 3.1 to get 
>>> DomainKeys stuff, and I feel like I've lost the SPF stuff. 
>>> Technology vendors everywhere are telling me that if I implement SPF 
>>> and DK that the entire plannet will be spam free.
>>> http://www.ranum.com/security/computer_security/papers/a1-firewall/index.html 
>>>
>>
>>
>> I've actually got a few of those IPS tools in different sizes.  The 
>> work really nice, except for the one I use with aluminum.  Head the 
>> "copper only" warning.
>>
>> Anyway, obsession or not, I'm sure you'd rather have correct results 
>> over incorrect results.
>>
>>
>>> The subdomain thing is causing them to assume that people are 
>>> publishing SPF records for the MTA systems as well as the domains. I 
>>> realize now that this is the "right" way to do it, but to be honest, 
>>> I think most people fell asleep before reading that part of the 
>>> spec. I know I did, and apparently, so did the folks over at google.
>>
>>
>> I recall falling asleep at least once too.  It's certainly not the 
>> clearest draft I've ever read.
>>
>>
>>> And if google can't get it right, there's no hope for me.
>>
>>
>> Google's got it right.  The host records are OPTIONAL and are only 
>> referred to as RECOMMENDED in the draft.  Google has correctly 
>> implemented the mandatory parts of the draft.
>>
>>
>> Daryl
>>
>

Re: SPF and Upgrade to SA 3.1

Posted by Ben Lentz <bl...@channing-bete.com>.
Yes- I would rather have correct results than just results.

Okay, so the problem is with my MTA moving the Return-Path header below 
the Received headers. It's breaking spamassassin's ability to check 
perfectly compliant SPF records. I'm using stock versions of sendmail 
8.13 on all my boxes, so I guess my only recourse is to go to the 
sendmail folks and ask them why.

It's probably not worth it- I'm really not in the mood to think about 
recompliling sendmail at the moment.

Thanks for your help.

----- Original Message -----
*From:* "Daryl C. W. O'Shea" <sp...@dostech.ca>
*Sent:* 09/29/2005 03:36:15 PM
*To:* Ben Lentz <bl...@channing-bete.com>
*Cc:* users@spamassassin.apache.org
*Subject:* SPF and Upgrade to SA 3.1



> Ben Lentz wrote:
>
>> _You_ are _welcome_.
>>
>> Get it moved? - Hmmm... Ala-kazamm! - Oh, that didn't work. Okay, so 
>> magic isn't going to get it moved, and I'm all out of ideas.
>
>
> I can only suggest starting another thread here or "somewhere else 
> applicable" that asks "this is the software I'm using, why is my 
> return-path being appended to the bottom and not prepended to the top?".
>
> If you're specific in the software you're using I'd be surprised if 
> someone can't give you a reason why and how to fix it.
>
>
>> I still don't understand why I used to get SPF_HELO_PASSes with 3.0.4 
>> and I don't with 3.1. The world hasn't changed, just my SA version. I 
>> guess SA was doing it "wrong" before, but is doing it "right" now? Is 
>> that the concrete explanation?
>
>
> YES!  You'll continue to get SPF_HELO_PASSes with SA 3.1.0 too, for 
> domains that fully implement the draft and publish SPF records for 
> each host.  SA 3.0 did INCORRECTLY fire SPF_HELO_PASSes for domains it 
> shouldn't have.  Surely you would prefer correct results and not just 
> results.
>
> The fact of the matter, though, is that nobody really cares about 
> SPF_HELO_* checks.  Some, usually smaller, domains publish them but 
> that's about it.
>
> Heck, even SA doesn't care about SPF_HELO_PASS:
>
> score SPF_HELO_PASS -0.001
>
>
> Also FWIW, I ran your test message through SA 3.0.4-r165054 and got 
> the incorrect SPF_HELO_PASS result.  I did NOT get an SPF_PASS result 
> though because:
>
> debug: Return-Path header found after 1 or more Received lines, cannot 
> trust envelope-from
>
>
>> The idea I'm obsessed with is that I moved to SA 3.1 to get 
>> DomainKeys stuff, and I feel like I've lost the SPF stuff. Technology 
>> vendors everywhere are telling me that if I implement SPF and DK that 
>> the entire plannet will be spam free.
>> http://www.ranum.com/security/computer_security/papers/a1-firewall/index.html 
>>
>
>
> I've actually got a few of those IPS tools in different sizes.  The 
> work really nice, except for the one I use with aluminum.  Head the 
> "copper only" warning.
>
> Anyway, obsession or not, I'm sure you'd rather have correct results 
> over incorrect results.
>
>
>> The subdomain thing is causing them to assume that people are 
>> publishing SPF records for the MTA systems as well as the domains. I 
>> realize now that this is the "right" way to do it, but to be honest, 
>> I think most people fell asleep before reading that part of the spec. 
>> I know I did, and apparently, so did the folks over at google.
>
>
> I recall falling asleep at least once too.  It's certainly not the 
> clearest draft I've ever read.
>
>
>> And if google can't get it right, there's no hope for me.
>
>
> Google's got it right.  The host records are OPTIONAL and are only 
> referred to as RECOMMENDED in the draft.  Google has correctly 
> implemented the mandatory parts of the draft.
>
>
> Daryl
>

Re: SPF and Upgrade to SA 3.1

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
Ben Lentz wrote:
> _You_ are _welcome_.
> 
> Get it moved? - Hmmm... Ala-kazamm! - Oh, that didn't work. Okay, so 
> magic isn't going to get it moved, and I'm all out of ideas.

I can only suggest starting another thread here or "somewhere else 
applicable" that asks "this is the software I'm using, why is my 
return-path being appended to the bottom and not prepended to the top?".

If you're specific in the software you're using I'd be surprised if 
someone can't give you a reason why and how to fix it.


> I still don't understand why I used to get SPF_HELO_PASSes with 3.0.4 
> and I don't with 3.1. The world hasn't changed, just my SA version. I 
> guess SA was doing it "wrong" before, but is doing it "right" now? Is 
> that the concrete explanation?

YES!  You'll continue to get SPF_HELO_PASSes with SA 3.1.0 too, for 
domains that fully implement the draft and publish SPF records for each 
host.  SA 3.0 did INCORRECTLY fire SPF_HELO_PASSes for domains it 
shouldn't have.  Surely you would prefer correct results and not just 
results.

The fact of the matter, though, is that nobody really cares about 
SPF_HELO_* checks.  Some, usually smaller, domains publish them but 
that's about it.

Heck, even SA doesn't care about SPF_HELO_PASS:

score SPF_HELO_PASS -0.001


Also FWIW, I ran your test message through SA 3.0.4-r165054 and got the 
incorrect SPF_HELO_PASS result.  I did NOT get an SPF_PASS result though 
because:

debug: Return-Path header found after 1 or more Received lines, cannot 
trust envelope-from


> The idea I'm obsessed with is that I 
> moved to SA 3.1 to get DomainKeys stuff, and I feel like I've lost the 
> SPF stuff. Technology vendors everywhere are telling me that if I 
> implement SPF and DK that the entire plannet will be spam free.
> http://www.ranum.com/security/computer_security/papers/a1-firewall/index.html

I've actually got a few of those IPS tools in different sizes.  The work 
really nice, except for the one I use with aluminum.  Head the "copper 
only" warning.

Anyway, obsession or not, I'm sure you'd rather have correct results 
over incorrect results.


> The subdomain thing is causing them to assume that people are publishing 
> SPF records for the MTA systems as well as the domains. I realize now 
> that this is the "right" way to do it, but to be honest, I think most 
> people fell asleep before reading that part of the spec. I know I did, 
> and apparently, so did the folks over at google.

I recall falling asleep at least once too.  It's certainly not the 
clearest draft I've ever read.


> And if google can't get it right, there's no hope for me.

Google's got it right.  The host records are OPTIONAL and are only 
referred to as RECOMMENDED in the draft.  Google has correctly 
implemented the mandatory parts of the draft.


Daryl


Re: SPF and Upgrade to SA 3.1

Posted by Ben Lentz <bl...@channing-bete.com>.
_You_ are _welcome_.

Get it moved? - Hmmm... Ala-kazamm! - Oh, that didn't work. Okay, so 
magic isn't going to get it moved, and I'm all out of ideas.

I still don't understand why I used to get SPF_HELO_PASSes with 3.0.4 
and I don't with 3.1. The world hasn't changed, just my SA version. I 
guess SA was doing it "wrong" before, but is doing it "right" now? Is 
that the concrete explanation? The idea I'm obsessed with is that I 
moved to SA 3.1 to get DomainKeys stuff, and I feel like I've lost the 
SPF stuff. Technology vendors everywhere are telling me that if I 
implement SPF and DK that the entire plannet will be spam free.
http://www.ranum.com/security/computer_security/papers/a1-firewall/index.html

The subdomain thing is causing them to assume that people are publishing 
SPF records for the MTA systems as well as the domains. I realize now 
that this is the "right" way to do it, but to be honest, I think most 
people fell asleep before reading that part of the spec. I know I did, 
and apparently, so did the folks over at google.

And if google can't get it right, there's no hope for me.

----- Original Message -----
*From:* "Daryl C. W. O'Shea" <sp...@dostech.ca>
*Sent:* 09/29/2005 02:33:05 PM
*To:* Ben Lentz <bl...@channing-bete.com>
*Cc:* users@spamassassin.apache.org
*Subject:* SPF and Upgrade to SA 3.1



> Ben Lentz wrote:
>
>> Here you go, the file has been _attached_.
>
>
> _thanks_. ;)
>
>
> The return-path header is in the wrong spot.  It should be the very 
> first line of the message.  Get it moved and you'll be set.
>
> Daryl
>

Re: SPF and Upgrade to SA 3.1

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
Ben Lentz wrote:
> Here you go, the file has been _attached_.

_thanks_. ;)


The return-path header is in the wrong spot.  It should be the very 
first line of the message.  Get it moved and you'll be set.

Daryl


Re: SPF and Upgrade to SA 3.1

Posted by Ben Lentz <bl...@channing-bete.com>.
Here you go, the file has been _attached_.

----- Original Message -----
*From:* "Daryl C. W. O'Shea" <sp...@dostech.ca>
*Sent:* 09/29/2005 12:32:08 PM
*To:* Ben Lentz <BL...@channing-bete.com>
*Cc:* users@spamassassin.apache.org
*Subject:* SPF and Upgrade to SA 3.1



> Ben Lentz wrote:
>
>> I'm lost. The email I received contains the header: "Return-Path: 
>> ben.lentz@gmail.com", so why would it be saying "spf: cannot get 
>> Envelope-From, cannot use SPF"?
>
>
> Usually it's a case of the header not being present during processing 
> and being added afterward.  If it's present when you scan manually 
> though, it should be detected.
>
> If you can _attach_ a copy of the message you're testing with and send 
> it to either myself or the list I'll take a look at it to see if I can 
> spot what's wrong.
>
> I'll be back in about three hours.
>
>
> Daryl
>

Re: SPF and Upgrade to SA 3.1

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
Ben Lentz wrote:
> I'm lost. The email I received contains the header: "Return-Path: 
> ben.lentz@gmail.com", so why would it be saying "spf: cannot get 
> Envelope-From, cannot use SPF"?

Usually it's a case of the header not being present during processing 
and being added afterward.  If it's present when you scan manually 
though, it should be detected.

If you can _attach_ a copy of the message you're testing with and send 
it to either myself or the list I'll take a look at it to see if I can 
spot what's wrong.

I'll be back in about three hours.


Daryl


Re: SPF and Upgrade to SA 3.1

Posted by Ben Lentz <BL...@channing-bete.com>.
I'm lost. The email I received contains the header: "Return-Path: 
ben.lentz@gmail.com", so why would it be saying "spf: cannot get 
Envelope-From, cannot use SPF"?

I appreciate your patience. What appeared to be something that broke in 
a SA version upgrade is not looking more like a FUBAR in my 
configuration and SPF record (mx vs. domain - including them both).

----- Original Message -----
*From:* "Daryl C. W. O'Shea" <sp...@dostech.ca>
*Sent:* 09/28/2005 9:54:27 PM -0400
*To:* Ben Lentz <BL...@channing-bete.com>
*Cc:* users@spamassassin.apache.org
*Subject:* SPF and Upgrade to SA 3.1



> Ben Lentz wrote:
>
>> Okay, I've added
>> always_trust_envelope_sender 1
>> trusted_networks 10.1.0.0/16
>> trusted_networks 205.246.7.107
>>
>> and restarted. Still not acknoledgement that SPF is working for 
>> gmail.com. SPF-based whitelisting might be great, but at this point 
>> I'm still not confident that SPF is working for me.
>>
>> Here's the debug info. The line "domain of sender wproxy.gmail.com 
>> does not designate mailers" leads me to believe that SA thinks 
>> gmail's SPF record is bad because they don't have any IN TXT 
>> wproxy.gmail.com SPF record, they're only publishing at the domain 
>> level.
>
>
> If you look at the dbg line before that you'll see that it's doing an 
> SPF_HELO check for wproxy.gmail.com. which doesn't have an SPF record 
> -- all is good here.
>
>> [28988] dbg: spf: checking HELO (helo=wproxy.gmail.com, 
>> ip=64.233.184.199)
>> [28988] dbg: spf: query for /64.233.184.199/wproxy.gmail.com: result: 
>> none, comment: SPF: domain of sender wproxy.gmail.com does not 
>> designate mailers
>
>
> All good (see above).
>
>
>> [28988] dbg: spf: cannot get Envelope-From, cannot use SPF
>
>
> Not good.  Make sure whatever is passing the mail to SpamAssassin is 
> including a Return-Path header.  This is a must.
>
>
> Daryl
>

Re: SPF and Upgrade to SA 3.1

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
Ben Lentz wrote:
> Okay, I've added
> always_trust_envelope_sender 1
> trusted_networks 10.1.0.0/16
> trusted_networks 205.246.7.107
> 
> and restarted. Still not acknoledgement that SPF is working for 
> gmail.com. SPF-based whitelisting might be great, but at this point I'm 
> still not confident that SPF is working for me.
> 
> Here's the debug info. The line "domain of sender wproxy.gmail.com does 
> not designate mailers" leads me to believe that SA thinks gmail's SPF 
> record is bad because they don't have any IN TXT wproxy.gmail.com SPF 
> record, they're only publishing at the domain level.

If you look at the dbg line before that you'll see that it's doing an 
SPF_HELO check for wproxy.gmail.com. which doesn't have an SPF record -- 
all is good here.

> [28988] dbg: spf: checking HELO (helo=wproxy.gmail.com, ip=64.233.184.199)
> [28988] dbg: spf: query for /64.233.184.199/wproxy.gmail.com: result: 
> none, comment: SPF: domain of sender wproxy.gmail.com does not designate 
> mailers

All good (see above).


> [28988] dbg: spf: cannot get Envelope-From, cannot use SPF

Not good.  Make sure whatever is passing the mail to SpamAssassin is 
including a Return-Path header.  This is a must.


Daryl


Re: SPF and Upgrade to SA 3.1

Posted by Ben Lentz <BL...@channing-bete.com>.
Okay, I've added
always_trust_envelope_sender 1
trusted_networks 10.1.0.0/16
trusted_networks 205.246.7.107

and restarted. Still not acknoledgement that SPF is working for 
gmail.com. SPF-based whitelisting might be great, but at this point I'm 
still not confident that SPF is working for me.

Here's the debug info. The line "domain of sender wproxy.gmail.com does 
not designate mailers" leads me to believe that SA thinks gmail's SPF 
record is bad because they don't have any IN TXT wproxy.gmail.com SPF 
record, they're only publishing at the domain level.

Thanks for all the help!

[28984] dbg: prefork: ordered 28988 to accept
[28988] info: spamd: connection from localhost.localdomain [127.0.0.1] 
at port 50129
[28984] dbg: prefork: child 28988: entering state 2
[28984] dbg: prefork: new lowest idle kid: 28989
[28988] warn: spamd: still running as root: user not specified with -u, 
not found, or set to root, falling back to nobody at /usr/bin/spamd line 
1150, <GEN8> line 3.
[28988] info: spamd: checking message <43...@gmail.com> for 
(unknown):99
[28988] dbg: dns: name server: 10.1.200.0, family: 2, ipv6: 0
[28988] dbg: received-header: parsed as [ ip=10.1.200.36 
rdns=smtp.channing-bete.com helo=smtp.channing-bete.com 
by=spam.channing-bete.com ident= envfrom= intl=0 id=j8T1cnxH028990 auth= ]
[28988] dbg: received-header: relay 10.1.200.36 trusted? yes internal? yes
[28988] dbg: received-header: parsed as [ ip=64.233.184.199 
rdns=wproxy.gmail.com helo=wproxy.gmail.com by=smtp.channing-bete.com 
ident= envfrom= intl=0 id=j8T1cgHY012157 auth= ]
[28988] dbg: received-header: relay 64.233.184.199 trusted? no internal? no
[28988] dbg: dns: looking up PTR record for '67.20.144.224'
[28988] dbg: dns: PTR for '67.20.144.224': ''
[28988] dbg: received-header: parsed as [ ip=67.20.144.224 rdns= helo= 
by=mx.gmail.com ident= envfrom= intl=0 
id=35sm106614wra.2005.09.28.18.38.50 auth= ]
[28988] dbg: received-header: relay 67.20.144.224 trusted? no internal? no
[28988] dbg: metadata: X-Spam-Relays-Trusted: [ ip=10.1.200.36 
rdns=smtp.channing-bete.com helo=smtp.channing-bete.com 
by=spam.channing-bete.com ident= envfrom= intl=1 id=j8T1cnxH028990 auth= ]
[28988] dbg: metadata: X-Spam-Relays-Untrusted: [ ip=64.233.184.199 
rdns=wproxy.gmail.com helo=wproxy.gmail.com by=smtp.channing-bete.com 
ident= envfrom= intl=0 id=j8T1cgHY012157 auth= ] [ ip=67.20.144.224 
rdns= helo= by=mx.gmail.com ident= envfrom= intl=0 
id=35sm106614wra.2005.09.28.18.38.50 auth= ]
[28988] dbg: metadata: X-Relay-Countries: US US
[28988] dbg: message: ---- MIME PARSER START ----
[28988] dbg: message: main message type: multipart/alternative
[28988] dbg: message: parsing multipart, got boundary: 
------------050703040406040403090403
[28988] dbg: message: found part of type text/plain, boundary: 
------------050703040406040403090403
[28988] dbg: message: parsing normal part
[28988] dbg: message: added part, type: text/plain
[28988] dbg: message: found part of type text/html, boundary: 
------------050703040406040403090403
[28988] dbg: message: parsing normal part
[28988] dbg: message: added part, type: text/html
[28988] dbg: message: ---- MIME PARSER END ----
[28988] dbg: message: decoding other encoding type (7bit), ignoring
[28988] dbg: message: decoding other encoding type (7bit), ignoring
[28988] dbg: textcat: message too short for language analysis
[28988] dbg: textcat: X-Languages: "", X-Languages-Length: 18
[28988] dbg: uridnsbl: domains to query:
[28988] dbg: dns: checking RBL sbl-xbl.spamhaus.org., set sblxbl-notfirsthop
[28988] dbg: dns: IPs found: full-external: 64.233.184.199, 
67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:
[28988] dbg: dns: only inspecting the following IPs: 64.233.184.199
[28988] dbg: dns: launching DNS A query for 
199.184.233.64.sbl-xbl.spamhaus.org. in background
[28988] dbg: dns: checking RBL sa-accredit.habeas.com., set 
habeas-firsttrusted
[28988] dbg: dns: IPs found: full-external: 64.233.184.199, 
67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:
[28988] dbg: dns: only inspecting the following IPs: 64.233.184.199
[28988] dbg: dns: launching DNS A query for 
199.184.233.64.sa-accredit.habeas.com. in background
[28988] dbg: dns: checking RBL sbl-xbl.spamhaus.org., set sblxbl
[28988] dbg: dns: IPs found: full-external: 64.233.184.199, 
67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:
[28988] dbg: dns: only inspecting the following IPs: 67.20.144.224, 
64.233.184.199
[28988] dbg: dns: launching DNS A query for 
224.144.20.67.sbl-xbl.spamhaus.org. in background
[28988] dbg: dns: checking RBL sa-other.bondedsender.org., set bsp-untrusted
[28988] dbg: dns: IPs found: full-external: 64.233.184.199, 
67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:
[28988] dbg: dns: only inspecting the following IPs: 67.20.144.224
[28988] dbg: dns: launching DNS TXT query for 
224.144.20.67.sa-other.bondedsender.org. in background
[28988] dbg: dns: checking RBL combined.njabl.org., set njabl-notfirsthop
[28988] dbg: dns: IPs found: full-external: 64.233.184.199, 
67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:
[28988] dbg: dns: only inspecting the following IPs: 64.233.184.199
[28988] dbg: dns: launching DNS A query for 
199.184.233.64.combined.njabl.org. in background
[28988] dbg: dns: checking RBL combined.njabl.org., set njabl
[28988] dbg: dns: IPs found: full-external: 64.233.184.199, 
67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:
[28988] dbg: dns: only inspecting the following IPs: 67.20.144.224, 
64.233.184.199
[28988] dbg: dns: launching DNS A query for 
224.144.20.67.combined.njabl.org. in background
[28988] dbg: dns: checking RBL 
combined-HIB.dnsiplists.completewhois.com., set whois
[28988] dbg: dns: IPs found: full-external: 64.233.184.199, 
67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:
[28988] dbg: dns: only inspecting the following IPs: 67.20.144.224, 
64.233.184.199
[28988] dbg: dns: launching DNS A query for 
224.144.20.67.combined-HIB.dnsiplists.completewhois.com. in background
[28988] dbg: dns: launching DNS A query for 
199.184.233.64.combined-HIB.dnsiplists.completewhois.com. in background
[28988] dbg: dns: checking RBL list.dsbl.org., set dsbl-notfirsthop
[28988] dbg: dns: IPs found: full-external: 64.233.184.199, 
67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:
[28988] dbg: dns: only inspecting the following IPs: 64.233.184.199
[28988] dbg: dns: launching DNS TXT query for 
199.184.233.64.list.dsbl.org. in background
[28988] dbg: dns: checking RBL bl.spamcop.net., set spamcop
[28988] dbg: dns: IPs found: full-external: 64.233.184.199, 
67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:
[28988] dbg: dns: only inspecting the following IPs: 67.20.144.224, 
64.233.184.199
[28988] dbg: dns: launching DNS TXT query for 
224.144.20.67.bl.spamcop.net. in background
[28988] dbg: dns: launching DNS TXT query for 
199.184.233.64.bl.spamcop.net. in background
[28988] dbg: dns: checking RBL sa-trusted.bondedsender.org., set 
bsp-firsttrusted
[28988] dbg: dns: IPs found: full-external: 64.233.184.199, 
67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:
[28988] dbg: dns: only inspecting the following IPs: 64.233.184.199
[28988] dbg: dns: launching DNS TXT query for 
199.184.233.64.sa-trusted.bondedsender.org. in background
[28988] dbg: dns: checking RBL 
combined-HIB.dnsiplists.completewhois.com., set whois-notfirsthop
[28988] dbg: dns: IPs found: full-external: 64.233.184.199, 
67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:
[28988] dbg: dns: only inspecting the following IPs: 64.233.184.199
[28988] dbg: dns: checking RBL dnsbl.sorbs.net., set sorbs-notfirsthop
[28988] dbg: dns: IPs found: full-external: 64.233.184.199, 
67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:
[28988] dbg: dns: only inspecting the following IPs: 64.233.184.199
[28988] dbg: dns: launching DNS A query for 
199.184.233.64.dnsbl.sorbs.net. in background
[28988] dbg: dns: checking RBL dnsbl.sorbs.net., set sorbs
[28988] dbg: dns: IPs found: full-external: 64.233.184.199, 
67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:
[28988] dbg: dns: only inspecting the following IPs: 67.20.144.224, 
64.233.184.199
[28988] dbg: dns: launching DNS A query for 
224.144.20.67.dnsbl.sorbs.net. in background
[28988] dbg: dns: checking RBL iadb.isipp.com., set iadb-firsttrusted
[28988] dbg: dns: IPs found: full-external: 64.233.184.199, 
67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:
[28988] dbg: dns: only inspecting the following IPs: 64.233.184.199
[28988] dbg: dns: launching DNS A query for 
199.184.233.64.iadb.isipp.com. in background
[28988] dbg: check: running tests for priority: 0
[28988] dbg: rules: running header regexp tests; score so far=0
[28988] dbg: rules: ran header rule __HAS_MSGID ======> got hit: "<"
[28988] dbg: rules: ran header rule __CT ======> got hit: "m"
[28988] dbg: rules: ran header rule __HAS_RCVD ======> got hit: "f"
[28988] dbg: rules: ran header rule __SANE_MSGID ======> got hit: 
"<43...@gmail.com>
[28988] dbg: rules: "
[28988] dbg: rules: ran header rule __MSGID_OK_HOST ======> got hit: 
"@gmail.com>"
[28988] dbg: rules: ran header rule __MIME_VERSION ======> got hit: "1"
[28988] dbg: rules: ran header rule __CTYPE_MULTIPART_ALT ======> got 
hit: "multipart/alternative"
[28988] dbg: rules: ran header rule __TOCC_EXISTS ======> got hit: "B"
[28988] dbg: rules: ran header rule __MOZILLA_MSGID ======> got hit: 
"<43...@gmail.com>"
[28988] dbg: rules: ran header rule __USER_AGENT ======> got hit: "M"
[28988] dbg: rules: ran header rule __HAS_SUBJECT ======> got hit: "t"
[28988] dbg: rules: ran header rule __CTYPE_HAS_BOUNDARY ======> got 
hit: "boundary"
[28988] dbg: rules: ran header rule __BAT_BOUNDARY ======> got hit: 
"boundary="----------"
[28988] dbg: spf: checking HELO (helo=wproxy.gmail.com, ip=64.233.184.199)
[28988] dbg: spf: query for /64.233.184.199/wproxy.gmail.com: result: 
none, comment: SPF: domain of sender wproxy.gmail.com does not designate 
mailers
[28988] dbg: eval: all '*From' addrs: ben.lentz@gmail.com
[28988] dbg: eval: trying Received header date for real time:  28 Sep 
2005 21:38:49 -0400
[28988] dbg: eval: time_t from date=1127957929, rcvd= 28 Sep 2005 
21:38:49 -0400
[28988] dbg: eval: trying Received header date for real time:  28 Sep 
2005 21:38:42 -0400
[28988] dbg: eval: time_t from date=1127957922, rcvd= 28 Sep 2005 
21:38:42 -0400
[28988] dbg: eval: trying Received header date for real time:  28 Sep 
2005 18:38:51 -0700
[28988] dbg: eval: time_t from date=1127957931, rcvd= 28 Sep 2005 
18:38:51 -0700
[28988] dbg: eval: trying Received header date for real time:  28 Sep 
2005 18:38:51 -0700
[28988] dbg: eval: time_t from date=1127957931, rcvd= 28 Sep 2005 
18:38:51 -0700
[28988] dbg: eval: trying Received header date for real time:  28 Sep 
2005 18:38:50 -0700
[28988] dbg: eval: time_t from date=1127957930, rcvd= 28 Sep 2005 
18:38:50 -0700
[28988] dbg: eval: all '*To' addrs: BLentz@channing-bete.com
[28988] dbg: spf: cannot get Envelope-From, cannot use SPF
[28988] dbg: eval: forged-HELO: from=gmail.com helo=gmail.com 
by=channing-bete.com
[28988] dbg: eval: forged-HELO: from= helo= by=gmail.com
[28988] dbg: rules: ran eval rule DK_SIGNED ======> got hit
[28988] dbg: rules: ran eval rule DK_VERIFIED ======> got hit
[28988] dbg: spf: def_spf_whitelist_from: could not find useable 
envelope sender
[28988] dbg: eval: date chosen from message: Wed Sep 28 21:38:42 2005
[28988] dbg: spf: spf_whitelist_from: could not find useable envelope sender
[28988] dbg: rules: running body-text per-line regexp tests; score so far=0
[28988] dbg: rules: ran body rule __NONEMPTY_BODY ======> got hit: "t"
[28988] dbg: uri: running uri tests; score so far=0
[28988] dbg: rules: ran eval rule __HTML_LENGTH_512 ======> got hit
[28988] dbg: bayes: not scoring message, returning undef
[28988] dbg: bayes: opportunistic call attempt failed, DB not readable
[28988] dbg: rules: ran eval rule __TAG_EXISTS_BODY ======> got hit
[28988] dbg: rules: ran eval rule __HTML_LENGTH_0000_1024 ======> got hit
[28988] dbg: rules: ran eval rule __MIME_HTML ======> got hit
[28988] dbg: rules: ran eval rule HTML_MESSAGE ======> got hit
[28988] dbg: rules: ran eval rule __HTML_LENGTH_384 ======> got hit
[28988] dbg: rules: ran eval rule __TAG_EXISTS_HTML ======> got hit
[28988] dbg: eval: text words: 1, html words: 1
[28988] dbg: eval: madiff: left: 0, orig: 1, max-difference: 0.00%
[28988] dbg: rules: ran eval rule __TAG_EXISTS_HEAD ======> got hit
[28988] dbg: rules: ran eval rule __TAG_EXISTS_META ======> got hit
[28988] dbg: rules: running raw-body-text per-line regexp tests; score 
so far=0.001
[28988] dbg: rules: running full-text regexp tests; score so far=0.001
[28988] dbg: info: entering helper-app run mode
[28988] dbg: info: leaving helper-app run mode
[28988] dbg: razor2: part=0 engine=4 contested=0 confidence=0
[28988] dbg: razor2: part=1 engine=4 contested=0 confidence=0
[28988] dbg: razor2: results: spam? 0
[28988] dbg: razor2: results: engine 8, highest cf score: 0
[28988] dbg: razor2: results: engine 4, highest cf score: 0
[28988] dbg: pyzor: pyzor is available: /usr/bin/pyzor
[28988] dbg: info: entering helper-app run mode
[28988] dbg: pyzor: opening pipe: /usr/bin/pyzor --homedir /etc/pyzor 
check < /tmp/.spamassassin28988HHDFz8tmp
[28992] dbg: util: changing real uid from 0 to match effective uid 99
[28992] dbg: util: setuid: ruid=99 euid=99
[28988] dbg: pyzor: [28992] finished:  exit=0x0100
[28988] dbg: pyzor: got response: 66.250.40.33:24441_(200, 'OK')_0_0
[28988] dbg: info: leaving helper-app run mode
[28988] dbg: dcc: dccifd is available: /etc/dcc/dccifd
[28988] dbg: info: entering helper-app run mode
[28988] dbg: dcc: dccifd got response: X-DCC-dcc.uncw.edu-Metrics: 
spam.channing-bete.com 1201; Body=1 Fuz1=6
[28988] dbg: info: leaving helper-app run mode
[28988] dbg: check: running tests for priority: 500
[28988] dbg: dns: success for 15 of 15 queries
[28988] dbg: rules: running meta tests; score so far=0.001
[28988] dbg: rules: running header regexp tests; score so far=0.001
[28988] dbg: rules: running body-text per-line regexp tests; score so 
far=0.001
[28988] dbg: uri: running uri tests; score so far=0.001
[28988] dbg: rules: running raw-body-text per-line regexp tests; score 
so far=0.001
[28988] dbg: rules: running full-text regexp tests; score so far=0.001
[28988] dbg: check: is spam? score=0.001 required=4
[28988] dbg: check: tests=DK_SIGNED,DK_VERIFIED,HTML_MESSAGE
[28988] dbg: check: 
subtests=__BAT_BOUNDARY,__CT,__CTYPE_HAS_BOUNDARY,__CTYPE_MULTIPART_ALT,__HAS_MSGID,__HAS_RCVD,__HAS_SUBJECT,__HTML_LENGTH_0000_1024,__HTML_LENGTH_384,__HTML_LENGTH_512,__MIME_HTML,__MIME_VERSION,__MOZILLA_MSGID,__MSGID_OK_HOST,__NONEMPTY_BODY,__RCVD_IN_NJABL,__RCVD_IN_SORBS,__SANE_MSGID,__TAG_EXISTS_BODY,__TAG_EXISTS_HEAD,__TAG_EXISTS_HTML,__TAG_EXISTS_META,__TOCC_EXISTS,__USER_AGENT
[28988] info: spamd: clean message (0.0/4.0) for (unknown):99 in 1.7 
seconds, 2230 bytes.
[28988] info: spamd: result: .  0 - DK_SIGNED,DK_VERIFIED,HTML_MESSAGE 
scantime=1.7,size=2230,user=(unknown),uid=99,required_score=4.0,rhost=localhost.localdomain,raddr=127.0.0.1,rport=50129,mid=<43...@gmail.com>,autolearn=disabled
[28988] dbg: config: copying current conf from backup
[28984] dbg: prefork: child 28988: entering state 1
[28984] dbg: prefork: new lowest idle kid: 28988
[28984] dbg: prefork: child reports idle
[28984] info: prefork: child states: II
[28988] dbg: prefork: sysread(7) not ready, wait max 300 secs


----- Original Message -----
*From:* "Daryl C. W. O'Shea" <sp...@dostech.ca>
*Sent:* 09/28/2005 9:34:57 PM -0400
*To:* Ben Lentz <BL...@channing-bete.com>
*Cc:* users@spamassassin.apache.org
*Subject:* SPF and Upgrade to SA 3.1



> Ben Lentz wrote:
>
>> Thanks for the info. I just added "always_trust_envelope_sender 1" to 
>> my local.cf and restarted. I then resent an email from gmail and 
>> still got no SPF. So, that didn't solve my problem.
>>
>> Am I incorrectly implimenting the standard? Do I need my TXT record 
>> to be located at IN TXT smtp.channing-bete.com -instead of- or -in 
>> addition to- a IN TXT channing-bete.com record?
>
>
> It would be both.  You'd want something like:
>
> channing-bete.com.      IN      TXT     "v=spf1 mx -all"
> smtp.channing-bete.com. IN      TXT     "v=spf1 a -all"
>
> You only need the first one for "regular" return-path based SPF 
> checks.  The second one only applies to HELO based SPF checks.
>
> In any case, although it's good to know, it doesn't apply to why 
> you're not seeing SPF results for mail from gmail.com (or likely 
> anyone else).
>
>
>> My internal setup might be the problem. I'd appreciate a 
>> recommendation if you have a moment... You may be able to see, based 
>> on the header I pasted and my DNS information, that my external MX is 
>> smtp.channing-bete.com (205.246.7.107), which is NATd by our firewall 
>> to 10.1.200.36. Mail is then routed internally to 
>> spam.channing-bete.com at 10.1.200.40 based on LDAP information 
>> (whether a given user is interested in being filtered through SA or 
>> not). This is where SpamAssassin is running, and where 3.0.4 used to 
>> be able to check records. :)
>
>
> I don't know why it would work with that setup in the past.  It 
> shouldn't have and didn't for any of my networks.
>
>
>> Therefore, my trusted_networks is 10.1.0.0/16. Correct? Or does it 
>> not make a difference? I've read that the gateway system *has* to be 
>> the one that does the SPF checking... and that's not the case in my 
>> setup. spam.channing-bete.com is one hop in from the external MX.
>
>
> Setting always_trust_envelope_sender to 1 removes the requirement for 
> SA to be on the first hop -- provided your trust path is set correctly.
>
> Setting trusted_networks correctly is the single most important step 
> in setting up SpamAssassin.  You need to include BOTH your external IP 
> and your internal IPs to trusted_networks:
>
> trusted_networks    10.1/16        # internal hosts
> trusted_networks    205.246.7.107    # public mx ip
>
>
> Lots of background, if you're interested, about trusted_networks and 
> what other things it affects here:
>
> http://wiki.apache.org/spamassassin/TrustPath
>
>
>> Honestly, I'd rather just run the former SA SPF checks on my system, 
>> crippled or inaccurate as they were, if they're not going to work 
>> with my configuration.
>
>
> The way regular SPF checks are done haven't changed from 3.0.  You 
> could probably use the old module but you'd almost definetely get the 
> same results you're getting now (along with a few Perl warnings).  
> You'd also miss out on SPF based whitelisting.
>
> The good news though is that 3.1's SPF plugin works just fine -- your 
> config is just a little incorrect.  I'm assuming you're missing the 
> second trusted_networks line (above)... adding that should fix your 
> problem.  If it doesn't, running the message in question through 
> spamassassin manually with some debug output will help.
>
> The output from both of the below would tell you a lot:
>
> spamassassin -Dspf < test.msg
> spamassassin -Dreceived-header < test.msg
>
>
> Daryl
>

Re: SPF and Upgrade to SA 3.1

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
Ben Lentz wrote:
> Thanks for the info. I just added "always_trust_envelope_sender 1" to my 
> local.cf and restarted. I then resent an email from gmail and still got 
> no SPF. So, that didn't solve my problem.
> 
> Am I incorrectly implimenting the standard? Do I need my TXT record to 
> be located at IN TXT smtp.channing-bete.com -instead of- or -in addition 
> to- a IN TXT channing-bete.com record?

It would be both.  You'd want something like:

channing-bete.com.      IN      TXT     "v=spf1 mx -all"
smtp.channing-bete.com. IN      TXT     "v=spf1 a -all"

You only need the first one for "regular" return-path based SPF checks. 
  The second one only applies to HELO based SPF checks.

In any case, although it's good to know, it doesn't apply to why you're 
not seeing SPF results for mail from gmail.com (or likely anyone else).


> My internal setup might be the problem. I'd appreciate a recommendation 
> if you have a moment... You may be able to see, based on the header I 
> pasted and my DNS information, that my external MX is 
> smtp.channing-bete.com (205.246.7.107), which is NATd by our firewall to 
> 10.1.200.36. Mail is then routed internally to spam.channing-bete.com at 
> 10.1.200.40 based on LDAP information (whether a given user is 
> interested in being filtered through SA or not). This is where 
> SpamAssassin is running, and where 3.0.4 used to be able to check 
> records. :)

I don't know why it would work with that setup in the past.  It 
shouldn't have and didn't for any of my networks.


> Therefore, my trusted_networks is 10.1.0.0/16. Correct? Or does it not 
> make a difference? I've read that the gateway system *has* to be the one 
> that does the SPF checking... and that's not the case in my setup. 
> spam.channing-bete.com is one hop in from the external MX.

Setting always_trust_envelope_sender to 1 removes the requirement for SA 
to be on the first hop -- provided your trust path is set correctly.

Setting trusted_networks correctly is the single most important step in 
setting up SpamAssassin.  You need to include BOTH your external IP and 
your internal IPs to trusted_networks:

trusted_networks	10.1/16		# internal hosts
trusted_networks	205.246.7.107	# public mx ip


Lots of background, if you're interested, about trusted_networks and 
what other things it affects here:

http://wiki.apache.org/spamassassin/TrustPath


> Honestly, I'd rather just run the former SA SPF checks on my system, 
> crippled or inaccurate as they were, if they're not going to work with 
> my configuration.

The way regular SPF checks are done haven't changed from 3.0.  You could 
probably use the old module but you'd almost definetely get the same 
results you're getting now (along with a few Perl warnings).  You'd also 
miss out on SPF based whitelisting.

The good news though is that 3.1's SPF plugin works just fine -- your 
config is just a little incorrect.  I'm assuming you're missing the 
second trusted_networks line (above)... adding that should fix your 
problem.  If it doesn't, running the message in question through 
spamassassin manually with some debug output will help.

The output from both of the below would tell you a lot:

spamassassin -Dspf < test.msg
spamassassin -Dreceived-header < test.msg


Daryl


Re: SPF and Upgrade to SA 3.1

Posted by Ben Lentz <BL...@channing-bete.com>.
Thanks for the info. I just added "always_trust_envelope_sender 1" to my 
local.cf and restarted. I then resent an email from gmail and still got 
no SPF. So, that didn't solve my problem.

Am I incorrectly implimenting the standard? Do I need my TXT record to 
be located at IN TXT smtp.channing-bete.com -instead of- or -in addition 
to- a IN TXT channing-bete.com record?

My internal setup might be the problem. I'd appreciate a recommendation 
if you have a moment... You may be able to see, based on the header I 
pasted and my DNS information, that my external MX is 
smtp.channing-bete.com (205.246.7.107), which is NATd by our firewall to 
10.1.200.36. Mail is then routed internally to spam.channing-bete.com at 
10.1.200.40 based on LDAP information (whether a given user is 
interested in being filtered through SA or not). This is where 
SpamAssassin is running, and where 3.0.4 used to be able to check 
records. :)

Therefore, my trusted_networks is 10.1.0.0/16. Correct? Or does it not 
make a difference? I've read that the gateway system *has* to be the one 
that does the SPF checking... and that's not the case in my setup. 
spam.channing-bete.com is one hop in from the external MX.

Honestly, I'd rather just run the former SA SPF checks on my system, 
crippled or inaccurate as they were, if they're not going to work with 
my configuration.

Thanks a million for the help so far.

----- Original Message -----
*From:* "Daryl C. W. O'Shea" <sp...@dostech.ca>
*Sent:* 09/28/2005 8:16:53 PM -0400
*To:* Ben Lentz <BL...@channing-bete.com>
*Cc:* users@spamassassin.apache.org
*Subject:* SPF and Upgrade to SA 3.1



> Ben Lentz wrote:
>
>> The message is sent from ben.lentz@gmail.com to 
>> blentz@channing-bete.com but shows up with no SPF information. Are 
>> you saying that the SPF records are supposed to be published along 
>> with the sending mail server's A record instead of with the domain? 
>> Like if the MX for channing-bete.com was smtp.channing-bete.com, then 
>> the SPF record should be returned from "dig smtp.channing-bete.com 
>> txt" and not "dig channing-bete.com txt"? This seems quite off from 
>> how gmail, yahoo, aol, microsoft, etc systems are publishing their 
>> records.
>
>
> (lots of background -- skip to the last line for the solution)
>
> To be consistent with the proposed standard (which hasn't changed in 
> this respect) ANY host that sends mail should have an SPF record if 
> you want it to pass an SPF_HELO_* check.  Further, EVERY host is 
> supposed to have a record if you want to publish that your 'other' 
> hosts shouldn't be sending mail.  I wouldn't worry too much about 
> SPF_HELO_* checks though.
>
>
> Regular SPF_* checks though, the ones that check against the 
> return-path address, can be a little more complicated (and are 
> actually a little broken in SA).
>
> For mail from the registered domain SA works properly, and checks the 
> SPF record found at IN TXT domain.com.  In the case of an address like 
> user@subdomain.domain.com SA (AFAIK) doesn't follow the standard and 
> doesn't check the SPF record which _could be_ at IN TXT 
> subdomain.domain.com. before checking the one at IN TXT domain.com.
>
>
> Anyway... all that really applies in your case is that SA should be 
> (if you tell it to) checking the SPF record found at IN TXT gmail.com. 
> for mail from ben.lentz@gmail.com.  The problem is you're not telling 
> it to check any SPF records (for non-helo checks).
>
>
>> This seemingly used to work so nicely! Can I swap back my 
>> SpamAssassin/Plugins/SPF.pm from SA 3.0.4?
>
>
> You seemingly either changed your network setup (moving your SA scan 
> from the first hop to the second) when upgraded to 3.1.0 or you 
> seemingly though it was working when it wasn't. ;)
>
> SpamAssassin will refuse to do SPF_* checks if the mail has been 
> passed through internal (to your network) relays -- just in case one 
> of these relays might alter the return-path (fetchmail, etc.).
>
> If you are sure that your mail relays will not alter the return-path 
> (most normal SMTP hops won't) you can force SA to act as you'd expect 
> it to by adding the following line to your local.cf file.
>
> always_trust_envelope_sender    1
>
>
> Daryl
>

Re: SPF and Upgrade to SA 3.1

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
Ben Lentz wrote:
> The message is sent from ben.lentz@gmail.com to blentz@channing-bete.com 
> but shows up with no SPF information. Are you saying that the SPF 
> records are supposed to be published along with the sending mail 
> server's A record instead of with the domain? Like if the MX for 
> channing-bete.com was smtp.channing-bete.com, then the SPF record should 
> be returned from "dig smtp.channing-bete.com txt" and not "dig 
> channing-bete.com txt"? This seems quite off from how gmail, yahoo, aol, 
> microsoft, etc systems are publishing their records.

(lots of background -- skip to the last line for the solution)

To be consistent with the proposed standard (which hasn't changed in 
this respect) ANY host that sends mail should have an SPF record if you 
want it to pass an SPF_HELO_* check.  Further, EVERY host is supposed to 
have a record if you want to publish that your 'other' hosts shouldn't 
be sending mail.  I wouldn't worry too much about SPF_HELO_* checks though.


Regular SPF_* checks though, the ones that check against the return-path 
address, can be a little more complicated (and are actually a little 
broken in SA).

For mail from the registered domain SA works properly, and checks the 
SPF record found at IN TXT domain.com.  In the case of an address like 
user@subdomain.domain.com SA (AFAIK) doesn't follow the standard and 
doesn't check the SPF record which _could be_ at IN TXT 
subdomain.domain.com. before checking the one at IN TXT domain.com.


Anyway... all that really applies in your case is that SA should be (if 
you tell it to) checking the SPF record found at IN TXT gmail.com. for 
mail from ben.lentz@gmail.com.  The problem is you're not telling it to 
check any SPF records (for non-helo checks).


> This seemingly used to work so nicely! Can I swap back my 
> SpamAssassin/Plugins/SPF.pm from SA 3.0.4?

You seemingly either changed your network setup (moving your SA scan 
from the first hop to the second) when upgraded to 3.1.0 or you 
seemingly though it was working when it wasn't. ;)

SpamAssassin will refuse to do SPF_* checks if the mail has been passed 
through internal (to your network) relays -- just in case one of these 
relays might alter the return-path (fetchmail, etc.).

If you are sure that your mail relays will not alter the return-path 
(most normal SMTP hops won't) you can force SA to act as you'd expect it 
to by adding the following line to your local.cf file.

always_trust_envelope_sender    1


Daryl


Re: SPF and Upgrade to SA 3.1

Posted by Ben Lentz <BL...@channing-bete.com>.
The message is sent from ben.lentz@gmail.com to blentz@channing-bete.com 
but shows up with no SPF information. Are you saying that the SPF 
records are supposed to be published along with the sending mail 
server's A record instead of with the domain? Like if the MX for 
channing-bete.com was smtp.channing-bete.com, then the SPF record should 
be returned from "dig smtp.channing-bete.com txt" and not "dig 
channing-bete.com txt"? This seems quite off from how gmail, yahoo, aol, 
microsoft, etc systems are publishing their records. So who's right? Did 
the draft standard change?

This seemingly used to work so nicely! Can I swap back my 
SpamAssassin/Plugins/SPF.pm from SA 3.0.4?

As requested, here's the header from a message sent from 
ben.lentz@gmail.com to blentz@channing-bete.com. Notice the lack of SPF 
tags, positive or negative, while gmail.com supposedly supports SPF.

Received: from spam.channing-bete.com ([10.1.200.40]) by sxgen01.dclb 
with Microsoft SMTPSVC(6.0.3790.211);
     Wed, 28 Sep 2005 19:30:26 -0400
Received: from smtp.channing-bete.com (smtp.channing-bete.com [10.1.200.36])
     by spam.channing-bete.com (8.13.1/8.13.1) with ESMTP id j8SNUK4s027770
     for <BL...@channing-bete.com>; Wed, 28 Sep 2005 19:30:20 -0400
Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.194])
     by smtp.channing-bete.com (8.13.1/8.13.1) with ESMTP id j8SNUEcu009571
     for <BL...@channing-bete.com>; Wed, 28 Sep 2005 19:30:14 -0400
Received: by wproxy.gmail.com with SMTP id i31so34974wra
     for <BL...@channing-bete.com>; Wed, 28 Sep 2005 16:30:13 -0700 (PDT)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
     s=beta; d=gmail.com;
     
h=received:message-id:date:from:user-agent:x-accept-language:mime-version:to:subject:content-type;
     
b=TqqEs8f1rd7hHxPdlueZld3eXiPU18zTv63O2i0mn5c9NQRRT5kJgpbdsDphJPyAC+tmxdDFXIlpjBizGEdFjPiKXk595iZ5HqJMWpobjBFImKdK+aZFYhWVqkPg8CRqW0ToBtNnRQSJNyrnFzbwdU99xoKZsU8xVYcVCOzNplo=
Received: by 10.54.122.14 with SMTP id u14mr195815wrc;
     Wed, 28 Sep 2005 16:30:13 -0700 (PDT)
Received: from ?192.0.0.11? ( [67.20.144.224])
     by mx.gmail.com with ESMTP id g7sm326139wra.2005.09.28.16.30.13;
     Wed, 28 Sep 2005 16:30:13 -0700 (PDT)
Message-ID: <43...@gmail.com>
Date: Wed, 28 Sep 2005 19:29:55 -0400
From: Ben Lentz <be...@gmail.com>
User-Agent: Mozilla Thunderbird 1.0.6 (Windows/20050716)
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: Ben Lentz <BL...@channing-bete.com>
Subject: test
Content-Type: multipart/alternative;
     boundary="------------060908000300050900050601"
X-Antivirus: Scanned by Vexira Antivirus 1.1.4
X-Spam-Flag: NO
X-Scanned-By: milter-spamc/0.25.321 (spam.channing-bete.com 
[10.1.200.40]); Wed, 28 Sep 2005 19:30:24 -0400
X-Spam-Status: NO, hits=0.00 required=4.00
X-Spam-Level:
X-Spam-Report: Content preview: test test [...]
     ____
     Content analysis details: (0.0 points, 4.0 required)
     ____
     pts rule name description
     ---- ---------------------- 
--------------------------------------------------
     0.0 DK_SIGNED Domain Keys: message has an unverified signature
     -0.0 DK_VERIFIED Domain Keys: signature passes verification
     0.0 HTML_MESSAGE BODY: HTML included in message
     ____
Return-Path: ben.lentz@gmail.com
X-OriginalArrivalTime: 28 Sep 2005 23:30:26.0881 (UTC) 
FILETIME=[9B384310:01C5C484]
       

----- Original Message -----
*From:* "Daryl C. W. O'Shea" <sp...@dostech.ca>
*Sent:* 09/28/2005 6:21:01 PM -0400
*To:* Ben Lentz <bl...@channing-bete.com>
*Cc:* users@spamassassin.apache.org
*Subject:* SPF and Upgrade to SA3.1



> Ben Lentz wrote:
>
>> Is my testing of gmail's information faulty? What has changed between 
>> 3.0.4 and 3.1.0 SPF-wise that would cause SPF to stop working?
>
>
> What's changed?  SPF_HELO_* checks are now correctly done -- against 
> host names and not registered domains.
>
> I think we -- correctly -- stopped stripping the sender's 
> (return-path) domain down too.
>
>
> Any examples you provide need to at least include the return-path 
> address and preferrably the raw headers.
>
>
> Daryl
>

Re: SPF and Upgrade to SA3.1

Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
Ben Lentz wrote:
> Is my testing of gmail's information faulty? What has changed between 
> 3.0.4 and 3.1.0 SPF-wise that would cause SPF to stop working?

What's changed?  SPF_HELO_* checks are now correctly done -- against 
host names and not registered domains.

I think we -- correctly -- stopped stripping the sender's (return-path) 
domain down too.


Any examples you provide need to at least include the return-path 
address and preferrably the raw headers.


Daryl