You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Wil Decius <wi...@gmail.com> on 2008/07/11 16:06:28 UTC

SPF-check works, but Whitelist-by-SPF does not

I'm trying to get Spamassassin local configuration setup to
whitleist-by-SPF.  The box, as delivered to me, runs Debian with

 spamassassin -V
  SpamAssassin version 3.2.5-r609689
    running on Perl version 5.8.8

In local.cf I've added
	whitelist_from_spf      *@technologyladder.com


Checking the "target" SPF record it looks OK.

dig TXT technologyladder.com +short
	"v=spf1 mx ip4:64.14.60.0/27 ip4:64.14.53.64/26 ip4:67.151.144.115/32
ip4:64.20.188.0/24 ip4:64.210.209.0/24 ip4:165.193.208.0/24
ip4:165.193.209.0/24 ip4:165.193.210.0/24 ip4:165.193.211.0/24 -all"

But email received FROM the "target" does NOT get whitelisted.


The message headers contain

	From: 	jobs@technologyladder.com
	
	Return-Path: 	<jo...@technologyladder.com>
	
	X-Spam-Report:
		*  1.5 FH_RELAY_NODNS We could not determine your Reverse DNS
		* -0.0 SPF_PASS SPF: sender matches SPF record
		*  5.0 BOTNET Relay might be a spambot or virusbot
		*      [botnet0.8,ip=165.193.208.162,rdns=r14nj3ip1.idc.technologyladder.com,maildomain=technologyladder.com,client,clientwords]
		*  0.0 HTML_MESSAGE BODY: HTML included in message
		*  1.5 BAYES_50 BODY: Bayesian spam probability is 40 to 60%
		*      [score: 0.4966]
		*  1.5 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
		*  0.3 AWL AWL: From: address is in the auto white-list
	
	Received: 	from r14nj3ip1.idc.technologyladder.com ([165.193.208.162]
verified) by mail.mydomain.com (SMTP) with ESMTP id 6850528 for
myuser@mydomain.com; Fri, 11 Jul 2008 02:28:10 -0700
	
	Received: 	from unknown (HELO script1.idc.theladders.com)
([10.0.1.221]) by r14nj3ip1.idc.technologyladder.com with ESMTP; 11
Jul 2008 05:28:08 -0400

What do I need to add/change so SPF Whitelisting works?

Wil

Re: SPF-check works, but Whitelist-by-SPF does not

Posted by Wil Decius <wi...@gmail.com>.
> That's the sitewide config file, though, and the debug output explicitly
> says "jobs@technologyladder.com is not in >>user's<<
> WHITELIST_FROM_SPF".

Ok.  I'm not sure what to do about that -- I'm only reporting what I see.

There is only ONE local.cf on this box.

Re: SPF-check works, but Whitelist-by-SPF does not

Posted by Wil Decius <wi...@gmail.com>.
> It's REALLY shooting in the dark beause it indicates a possibly broken
> parser, but, I'm inclined to suggest that as quoted he has a lead blank
> in front of his whitelist_from_spf entry.
>
> I'd also retype it to make sre there are no messed up characters line
> a 0xa0 space in there. (I also found in the dark past and avoid it through
> today that <tab><spc><spc> for pretty printing was bad usage. The parser
> was unhappy with it. I don't remember if it turned up as a lint error or
> not. I think it was something I tried in desperation.)

I deleted then retyped the entire line.

Still, no change -- and no WHITELISTing.

Maddening ...

Re: SPF-check works, but Whitelist-by-SPF does not

Posted by mouss <mo...@netoyen.net>.
jdow wrote:
> From: "John Hardin" <jh...@impsec.org>
> Sent: Monday, 2008, July 14 09:30
>>
>> On Mon, 2008-07-14 at 08:14 -0700, Wil Decius wrote:
>>> > Are you sure you're checking the correct config file?
>>>
>>> Yes.  From the debug output,
>>>
>>> [7596] dbg: config: using "/etc/mail/spamassassin" for site rules dir
>>> [7596] dbg: config: read file /etc/mail/spamassassin/local.cf
>>>
>>> Which is the file I'm editing.
>>
>> That's the sitewide config file, though, and the debug output explicitly
>> says "jobs@technologyladder.com is not in >>user's<<
>> WHITELIST_FROM_SPF".
>
> It implied both the system and the user.
>
> [7562] dbg: spf: def_whitelist_from_spf: jobs@technologyladder.com is
> not in DEF_WHITELIST_FROM_SPF
> [7562] dbg: spf: whitelist_from_spf: jobs@technologyladder.com is not
> in user's WHITELIST_FROM_SPF

it implies the default and the user, but not the site.

>
> It's REALLY shooting in the dark beause it indicates a possibly broken
> parser, but, I'm inclined to suggest that as quoted he has a lead blank
> in front of his whitelist_from_spf entry.
>
> I'd also retype it to make sre there are no messed up characters line
> a 0xa0 space in there. (I also found in the dark past and avoid it 
> through
> today that <tab><spc><spc> for pretty printing was bad usage. The parser
> was unhappy with it. I don't remember if it turned up as a lint error or
> not. I think it was something I tried in desperation.)
>
> {^_^}


Re: SPF-check works, but Whitelist-by-SPF does not

Posted by jdow <jd...@earthlink.net>.
From: "John Hardin" <jh...@impsec.org>
Sent: Monday, 2008, July 14 09:30
> 
> On Mon, 2008-07-14 at 08:14 -0700, Wil Decius wrote:
>> > Are you sure you're checking the correct config file?
>> 
>> Yes.  From the debug output,
>> 
>> [7596] dbg: config: using "/etc/mail/spamassassin" for site rules dir
>> [7596] dbg: config: read file /etc/mail/spamassassin/local.cf
>> 
>> Which is the file I'm editing.
> 
> That's the sitewide config file, though, and the debug output explicitly
> says "jobs@technologyladder.com is not in >>user's<<
> WHITELIST_FROM_SPF".

It implied both the system and the user.

[7562] dbg: spf: def_whitelist_from_spf: jobs@technologyladder.com is
not in DEF_WHITELIST_FROM_SPF
[7562] dbg: spf: whitelist_from_spf: jobs@technologyladder.com is not
in user's WHITELIST_FROM_SPF

It's REALLY shooting in the dark beause it indicates a possibly broken
parser, but, I'm inclined to suggest that as quoted he has a lead blank
in front of his whitelist_from_spf entry.

I'd also retype it to make sre there are no messed up characters line
a 0xa0 space in there. (I also found in the dark past and avoid it through
today that <tab><spc><spc> for pretty printing was bad usage. The parser
was unhappy with it. I don't remember if it turned up as a lint error or
not. I think it was something I tried in desperation.)

{^_^}

Re: SPF-check works, but Whitelist-by-SPF does not

Posted by John Hardin <jh...@impsec.org>.
On Mon, 2008-07-14 at 08:14 -0700, Wil Decius wrote:
> > Are you sure you're checking the correct config file?
> 
> Yes.  From the debug output,
> 
> [7596] dbg: config: using "/etc/mail/spamassassin" for site rules dir
> [7596] dbg: config: read file /etc/mail/spamassassin/local.cf
> 
> Which is the file I'm editing.

That's the sitewide config file, though, and the debug output explicitly
says "jobs@technologyladder.com is not in >>user's<<
WHITELIST_FROM_SPF".

-- 
 John Hardin KA7OHZ                    http://www.impsec.org/~jhardin/
 jhardin@impsec.org    FALaholic #11174     pgpk -a jhardin@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
  ...every time I sit down in front of a Windows machine I feel as
  if the computer is just a place for the manufacturers to put their
  advertising.                                -- fwadling on Y! SCOX
----------------------------------------------------------------------
 2 days until the 63rd anniversary of the dawn of the Atomic Age


Re: SPF-check works, but Whitelist-by-SPF does not

Posted by Wil Decius <wi...@gmail.com>.
> Are you sure you're checking the correct config file?

Yes.  From the debug output,

[7596] dbg: config: using "/etc/mail/spamassassin" for site rules dir
[7596] dbg: config: read file /etc/mail/spamassassin/local.cf

Which is the file I'm editing.

In any case, other changes to it get picked up correctly.

Re: SPF-check works, but Whitelist-by-SPF does not

Posted by John Hardin <jh...@impsec.org>.
On Mon, 2008-07-14 at 06:49 -0700, Wil Decius wrote:

> [7562] dbg: spf: whitelist_from_spf: jobs@technologyladder.com is not
> in user's WHITELIST_FROM_SPF
----^^^^^^^

> but checking again in "local.cf"
> 
> grep technologyladder.com local.cf
>  whitelist_from_spf      *@technologyladder.com
> 
> Apparently that's not getting picked up.

Are you sure you're checking the correct config file?


-- 
 John Hardin KA7OHZ                    http://www.impsec.org/~jhardin/
 jhardin@impsec.org    FALaholic #11174     pgpk -a jhardin@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
  Usually Microsoft doesn't develop products, we buy products.
                          -- Arno Edelmann, Microsoft product manager
-----------------------------------------------------------------------
 2 days until the 63rd anniversary of the dawn of the Atomic Age


Re: SPF-check works, but Whitelist-by-SPF does not

Posted by Wil Decius <wi...@gmail.com>.
> Just for S&Gs, try changing that "whitelist_from_spf  *@technologyladder.com"
> to a "def_whitelist_from_spf *@technologyladder.com"

That made some sense to me to try!  But, unfortunately, still no change.

> One other possibility, it may be due to issues parsing the
> "Received:" header that your MTA adds. It doesn't seem to be in the
> standard format:
.
> So it may be an SA parse issue.

I haven't a clue - yet - as to how to check or how to fix that.

Re: SPF-check works, but Whitelist-by-SPF does not

Posted by David B Funk <db...@engineering.uiowa.edu>.
On Mon, 14 Jul 2008, Wil Decius wrote:

> > Try adding
> >
> > envelope_sender_header Return-Path
>
> I added that as well -- no difference.
>
> Using the "manual debug" output from above, I noticed,
>
> [7562] dbg: spf: def_whitelist_from_spf: jobs@technologyladder.com is
> not in DEF_WHITELIST_FROM_SPF
> [7562] dbg: spf: whitelist_from_spf: jobs@technologyladder.com is not
> in user's WHITELIST_FROM_SPF
>
> but checking again in "local.cf"
>
> grep technologyladder.com local.cf
>  whitelist_from_spf      *@technologyladder.com
>
> Apparently that's not getting picked up.
>
> Why?

For what ever reason it looks like your SA is looking for a
"def_whitelist_from_spf" -not- "whitelist_from_spf" in the system
local.cf file.
Just for S&Gs, try changing that "whitelist_from_spf  *@technologyladder.com"
to a "def_whitelist_from_spf *@technologyladder.com"

One other possibility, it may be due to issues parsing the
"Received:" header that your MTA adds. It doesn't seem to be in the
standard format:

   Received:       from r14nj3ip1.idc.technologyladder.com
 ([165.193.208.162] verified) by mail.mydomain.com (SMTP) with ESMTP id
 6850528 for myuser@mydomain.com; Fri, 11 Jul 2008 02:28:10 -0700

Usual MTA header says something like:
  Received:       from r14nj3ip1.idc.technologyladder.com
  (r14nj3ip1.idc.technologyladder.com [165.193.208.162]) by ...

(rather than saying "verified)" usual usage is to put the FQSN inside
the '()' along with the IP address).

So it may be an SA parse issue.

-- 
Dave Funk                                  University of Iowa
<dbfunk (at) engineering.uiowa.edu>        College of Engineering
319/335-5751   FAX: 319/384-0549           1256 Seamans Center
Sys_admin/Postmaster/cell_admin            Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{

Re: SPF-check works, but Whitelist-by-SPF does not

Posted by Wil Decius <wi...@gmail.com>.
> Try adding
>
> envelope_sender_header Return-Path

I added that as well -- no difference.

Using the "manual debug" output from above, I noticed,

[7562] dbg: spf: def_whitelist_from_spf: jobs@technologyladder.com is
not in DEF_WHITELIST_FROM_SPF
[7562] dbg: spf: whitelist_from_spf: jobs@technologyladder.com is not
in user's WHITELIST_FROM_SPF

but checking again in "local.cf"

grep technologyladder.com local.cf
 whitelist_from_spf      *@technologyladder.com

Apparently that's not getting picked up.

Why?

Re: SPF-check works, but Whitelist-by-SPF does not

Posted by Henrik K <he...@hege.li>.
On Mon, Jul 14, 2008 at 06:08:30AM -0700, Wil Decius wrote:
> > Answering my own observation this seems to work with my fetchmail based
> > system:
> >
> > always_trust_envelope_sender 1
> 
> Adding that line to my local.cf has no apparent effect -- at least on
> messages from the sender.  They're still Passing SPF, but NOT getting
> whitelisted.

Try adding

envelope_sender_header Return-Path


Re: SPF-check works, but Whitelist-by-SPF does not

Posted by Wil Decius <wi...@gmail.com>.
> Answering my own observation this seems to work with my fetchmail based
> system:
>
> always_trust_envelope_sender 1

Adding that line to my local.cf has no apparent effect -- at least on
messages from the sender.  They're still Passing SPF, but NOT getting
whitelisted.

Re: SPF-check works, but Whitelist-by-SPF does not

Posted by jdow <jd...@earthlink.net>.
From: "jdow" <jd...@earthlink.net>
Sent: Monday, 2008, July 14 01:18


> From: "Wil Decius" <wi...@gmail.com>
> Sent: Sunday, 2008, July 13 18:20
>
>
>>> if mail contains Received headers indicating that mail was forwarded by 
>>> a
>>> trusted hop (a hop is not necessarily a box. it may be a proxy, an MTA
>>> instance, ... etc), then addresses may have been rewritten and are thus
>>> "untrusted".
>>
>> On my box there's only my mail server & SA.  No proxies etc.  IIUC
>> from reading, there are no hops -- trusted or otherwise -- on my
>> server.
>>
>> Or are you suggesting that there's something at issue on the sender's
>> end of the transaction, given the headers above?  If so, is there a
>> less-global way to trust just THAT sender, rather than what I presume
>> to be SA-wide "always_trust_envelope_sender=1"?
>
> There is a basic problem with that line - it fails to parse.

Answering my own observation this seems to work with my fetchmail based
system:

always_trust_envelope_sender 1

{^_^} 


Re: SPF-check works, but Whitelist-by-SPF does not

Posted by jdow <jd...@earthlink.net>.
From: "Wil Decius" <wi...@gmail.com>
Sent: Sunday, 2008, July 13 18:20


>> if mail contains Received headers indicating that mail was forwarded by a
>> trusted hop (a hop is not necessarily a box. it may be a proxy, an MTA
>> instance, ... etc), then addresses may have been rewritten and are thus
>> "untrusted".
>
> On my box there's only my mail server & SA.  No proxies etc.  IIUC
> from reading, there are no hops -- trusted or otherwise -- on my
> server.
>
> Or are you suggesting that there's something at issue on the sender's
> end of the transaction, given the headers above?  If so, is there a
> less-global way to trust just THAT sender, rather than what I presume
> to be SA-wide "always_trust_envelope_sender=1"?

There is a basic problem with that line - it fails to parse.

{^_-} 


Re: SPF-check works, but Whitelist-by-SPF does not

Posted by Wil Decius <wi...@gmail.com>.
> if mail contains Received headers indicating that mail was forwarded by a
> trusted hop (a hop is not necessarily a box. it may be a proxy, an MTA
> instance, ... etc), then addresses may have been rewritten and are thus
> "untrusted".

On my box there's only my mail server & SA.  No proxies etc.  IIUC
from reading, there are no hops -- trusted or otherwise -- on my
server.

Or are you suggesting that there's something at issue on the sender's
end of the transaction, given the headers above?  If so, is there a
less-global way to trust just THAT sender, rather than what I presume
to be SA-wide "always_trust_envelope_sender=1"?

Re: SPF-check works, but Whitelist-by-SPF does not

Posted by mouss <mo...@netoyen.net>.
Wil Decius wrote:
> On Sun, Jul 13, 2008 at 3:30 PM, mouss <mo...@netoyen.net> wrote:
>   
>> If mail is forwarded by a trusted hop before SA, you need
>>
>> always_trust_envelope_sender=1
>>     
>
> There's nothing special about this SA installation.  It's just SA
> running @ "mail.mydomain.com".  I'm doing no forwarding from one box
> to another.  Rather, I'm just trying to "receive" simple mail.
>   

if mail contains Received headers indicating that mail was forwarded by 
a trusted hop (a hop is not necessarily a box. it may be a proxy, an MTA 
instance, ... etc), then addresses may have been rewritten and are thus 
"untrusted".
> Can you explain further what you had in mind here?
>   


Re: SPF-check works, but Whitelist-by-SPF does not

Posted by Wil Decius <wi...@gmail.com>.
On Sun, Jul 13, 2008 at 3:30 PM, mouss <mo...@netoyen.net> wrote:
> If mail is forwarded by a trusted hop before SA, you need
>
> always_trust_envelope_sender=1

There's nothing special about this SA installation.  It's just SA
running @ "mail.mydomain.com".  I'm doing no forwarding from one box
to another.  Rather, I'm just trying to "receive" simple mail.

Can you explain further what you had in mind here?

Re: SPF-check works, but Whitelist-by-SPF does not

Posted by mouss <mo...@netoyen.net>.
Wil Decius wrote:
> I'm trying to get Spamassassin local configuration setup to
> whitleist-by-SPF.  The box, as delivered to me, runs Debian with
>
>  spamassassin -V
>   SpamAssassin version 3.2.5-r609689
>     running on Perl version 5.8.8
>
> In local.cf I've added
> 	whitelist_from_spf      *@technologyladder.com
>
>
> Checking the "target" SPF record it looks OK.
>
> dig TXT technologyladder.com +short
> 	"v=spf1 mx ip4:64.14.60.0/27 ip4:64.14.53.64/26 ip4:67.151.144.115/32
> ip4:64.20.188.0/24 ip4:64.210.209.0/24 ip4:165.193.208.0/24
> ip4:165.193.209.0/24 ip4:165.193.210.0/24 ip4:165.193.211.0/24 -all"
>
> But email received FROM the "target" does NOT get whitelisted.
>
>
> The message headers contain
>
> 	From: 	jobs@technologyladder.com
> 	
> 	Return-Path: 	<jo...@technologyladder.com>
> 	
> 	X-Spam-Report:
> 		*  1.5 FH_RELAY_NODNS We could not determine your Reverse DNS
> 		* -0.0 SPF_PASS SPF: sender matches SPF record
> 		*  5.0 BOTNET Relay might be a spambot or virusbot
> 		*      [botnet0.8,ip=165.193.208.162,rdns=r14nj3ip1.idc.technologyladder.com,maildomain=technologyladder.com,client,clientwords]
> 		*  0.0 HTML_MESSAGE BODY: HTML included in message
> 		*  1.5 BAYES_50 BODY: Bayesian spam probability is 40 to 60%
> 		*      [score: 0.4966]
> 		*  1.5 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
> 		*  0.3 AWL AWL: From: address is in the auto white-list
> 	
> 	Received: 	from r14nj3ip1.idc.technologyladder.com ([165.193.208.162]
> verified) by mail.mydomain.com (SMTP) with ESMTP id 6850528 for
> myuser@mydomain.com; Fri, 11 Jul 2008 02:28:10 -0700
> 	
> 	Received: 	from unknown (HELO script1.idc.theladders.com)
> ([10.0.1.221]) by r14nj3ip1.idc.technologyladder.com with ESMTP; 11
> Jul 2008 05:28:08 -0400
>
> What do I need to add/change so SPF Whitelisting works?
>
> Wil
>   

If mail is forwarded by a trusted hop before SA, you need

always_trust_envelope_sender=1



Re: SPF-check works, but Whitelist-by-SPF does not

Posted by ram <ra...@netcore.co.in>.
On Fri, 2008-07-11 at 07:06 -0700, Wil Decius wrote:
> I'm trying to get Spamassassin local configuration setup to
> whitleist-by-SPF.  The box, as delivered to me, runs Debian with
> 
>  spamassassin -V
>   SpamAssassin version 3.2.5-r609689
>     running on Perl version 5.8.8
> 
> In local.cf I've added
> 	whitelist_from_spf      *@technologyladder.com
> 
> 
> Checking the "target" SPF record it looks OK.
> 
> dig TXT technologyladder.com +short
> 	"v=spf1 mx ip4:64.14.60.0/27 ip4:64.14.53.64/26 ip4:67.151.144.115/32
> ip4:64.20.188.0/24 ip4:64.210.209.0/24 ip4:165.193.208.0/24
> ip4:165.193.209.0/24 ip4:165.193.210.0/24 ip4:165.193.211.0/24 -all"
> 
> But email received FROM the "target" does NOT get whitelisted.
> 
> 
> The message headers contain
> 
> 	From: 	jobs@technologyladder.com
> 	
> 	Return-Path: 	<jo...@technologyladder.com>
> 	
> 	X-Spam-Report:
> 		*  1.5 FH_RELAY_NODNS We could not determine your Reverse DNS
> 		* -0.0 SPF_PASS SPF: sender matches SPF record
> 		*  5.0 BOTNET Relay might be a spambot or virusbot
> 		*      [botnet0.8,ip=165.193.208.162,rdns=r14nj3ip1.idc.technologyladder.com,maildomain=technologyladder.com,client,clientwords]
> 		*  0.0 HTML_MESSAGE BODY: HTML included in message
> 		*  1.5 BAYES_50 BODY: Bayesian spam probability is 40 to 60%
> 		*      [score: 0.4966]
> 		*  1.5 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
> 		*  0.3 AWL AWL: From: address is in the auto white-list
> 	
> 	Received: 	from r14nj3ip1.idc.technologyladder.com ([165.193.208.162]
> verified) by mail.mydomain.com (SMTP) with ESMTP id 6850528 for
> myuser@mydomain.com; Fri, 11 Jul 2008 02:28:10 -0700
> 	
> 	Received: 	from unknown (HELO script1.idc.theladders.com)
> ([10.0.1.221]) by r14nj3ip1.idc.technologyladder.com with ESMTP; 11
> Jul 2008 05:28:08 -0400



Just simulate a message that should spf-pass & whitelist 

then run 
spamassassin -D -t < FILE.eml 

What is the output you get, that info would help