You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Gino Cerullo <gc...@pixelpointstudios.com> on 2006/07/10 22:26:53 UTC
Spamassassin and SPF
I have a question regarding Spamassassin's behaviour with SPF.
My configuration is running Postfix v2.1.5 and Spamassassin v3.0.1 is
called by Amavis-new v2.2.0 as is installed as part of Mac OS X
Server 10.4.x
All domains handled by the email server have strict SPF records that
require all account holders to send their email through the server.
As such all account holders authenticate with the server through the
submission port 587 when they want to send email from outside the
network.
I noticed that Spamassassin is scoring an SPF_FAIL with these users
based on the fact that they are connecting with IPs that fall outside
what is in their domain's SPF record.
It seems to me that this is the incorrect behaviour. Spamassassin
should not be evaluating the IP address from which the clients are
connecting but should be ignoring that altogether.
The Postfix policy daemon is handling this correctly , as it should,
ignoring where the client is connecting from since it is the outbound
server. It's not evaluating itself against the SPF record.
Is there something that is configured incorrectly that is causing
this to happen? Has this behaviour changed in a more recent version
of Spamassassin?
Any suggestions would be appritiated.
--
Gino Cerullo
Pixel Point Studios
21 Chesham Drive
Toronto, ON M3M 1W6
T: 416-247-7740
F: 416-247-7503
Re: Spamassassin and SPF
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 7/10/2006 9:57 PM, Gino Cerullo wrote:
>
> On 10-Jul-06, at 9:16 PM, Daryl C. W. O'Shea wrote:
>
> Snip, snip and more snip.
>> Also note that SPF isn't the only thing suffering from your trust
>> path issues, it's just a symptom you've noticed. You'll also
>> currently be doing dynablock checks against users you'd rather not
>> be, along with a whole slew of other tests that will FP when SA
>> thinks it's testing mail from some random system/zombie and not an
>> authenticated user.
>>
>
> So what you're saying is that I'm better off not scanning authenticated
> users. I'll take your word on that.
There's probably just as many advantages as there are disadvantages for
doing it either way.
If you can't inform SA of a user's auth status then you've got to skip
the SA check. If you can provide the auth info, then SA will work fine
(if it supports parsing of your particular auth info), and it's a matter
of personal preference.
>> Let me know if you're running Postfix 2.3 and can enable the auth
>> headers in your config. I'll probably get to making a patch tonight
>> as long as the rain doesn't stop and I don't get distracted by the
>> big stash of fireworks I've accumulated. :)
>>
>
> Well I'm running Postfix v2.1.3 (standard in OS X Server 10.4.x.) I'm
> waiting until Apple previews OS X 10.5 in August to see whether 10.5
> includes Postfix v2.3. If not than I may do the upgrade myself in 10.4.
AFAIK, v2.3 is the first to support adding auth headers, so, yeah,
unless you upgrade you'll have to go with the first option.
> Since SA is being called by Amavisd-new shouldn't the changes to ignore
> authenticated user happen there? I think I read that somewhere, maybe
> the Amavis mailing list. That's the problem with being subscribed to
> all these lists. They all start to run into each other in your head.
> Off to the archives I go.
I know of, but am not at all familiar with Amavisd-new configuration, so
I have no idea. Do what works.
> BIG STASH OF FIREWORKS! Boy I'm glad you don't live in my
> neighbourhood. :-O
My closest neighbor is about 4 km away so I don't have to worry about
blowing up the neighborhood. I do, however, need lots of fireworks if I
want to annoy them. :)
Daryl
Re: Spamassassin and SPF
Posted by Gino Cerullo <gc...@pixelpointstudios.com>.
On 10-Jul-06, at 9:16 PM, Daryl C. W. O'Shea wrote:
Snip, snip and more snip.
Thanks for all that good info.
>> I see from the header in the message you sent that you have
>> deployed DKIM. I'm hoping to do that as well but not for a while
>> yet. Do similar problems arise with DKIM and SA as we've
>> discussed here with SPF?
>
> DKIM doesn't rely on any defined set of relays. It uses the
> envelope sender (usually just the domain) and the signature found
> in the headers.
>
Someday I'll have some time to play with this and get a better
understanding of DKIM.
>
> Also note that SPF isn't the only thing suffering from your trust
> path issues, it's just a symptom you've noticed. You'll also
> currently be doing dynablock checks against users you'd rather not
> be, along with a whole slew of other tests that will FP when SA
> thinks it's testing mail from some random system/zombie and not an
> authenticated user.
>
So what you're saying is that I'm better off not scanning
authenticated users. I'll take your word on that.
>
> Let me know if you're running Postfix 2.3 and can enable the auth
> headers in your config. I'll probably get to making a patch
> tonight as long as the rain doesn't stop and I don't get distracted
> by the big stash of fireworks I've accumulated. :)
>
Well I'm running Postfix v2.1.3 (standard in OS X Server 10.4.x.) I'm
waiting until Apple previews OS X 10.5 in August to see whether 10.5
includes Postfix v2.3. If not than I may do the upgrade myself in 10.4.
Since SA is being called by Amavisd-new shouldn't the changes to
ignore authenticated user happen there? I think I read that
somewhere, maybe the Amavis mailing list. That's the problem with
being subscribed to all these lists. They all start to run into each
other in your head. Off to the archives I go.
BIG STASH OF FIREWORKS! Boy I'm glad you don't live in my
neighbourhood. :-O
>
> Daryl
>
--
Gino Cerullo
Pixel Point Studios
21 Chesham Drive
Toronto, ON M3M 1W6
T: 416-247-7740
F: 416-247-7503
Re: Spamassassin and SPF
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 7/10/2006 8:47 PM, Gino Cerullo wrote:
>
> On 10-Jul-06, at 7:39 PM, Daryl C. W. O'Shea wrote:
>> You have two options:
>>
>> 1) Quoting mouss...
>>
>> mouss wrote:
>> > martin f krafft a écrit :
>> >> Well, sure, this makes sense, but how can I support this standard
>> >> use-case? Postfix adding a SASL-header that causes Spamassassin then
>> >> to ignore the message isn't the solution as spammers would simply do
>> >> that sooner or later. Short of whitelisting people, what should
>> >> I do?
>> >>
>> >
>> > how do you integrate SA with postfix?
>> >
>> > If using content_filter, then you could skip SA for authenticated
>> users.
<snip config details>
> I see but is the trade-off here that if SA skips authenticated users
> and they are compromised then they can become spam sources that
> wouldn't be caught locally or does it only skip SPF and still do all
> other scans?
Skipping SA for auth'd users in your Postfix config would skip all of
SA, not just SPF checks in SA.
I wouldn't necessarily call not spam checking local spam zombies a
trade-off. Personally, I'd want to receive the spam from these local
machines/users so that I can take action against/for those users.
>> or
>>
>> 2) Upgrade to Postfix 2.3, if necessary, set
>> "smtpd_sasl_authenticated_header yes" in your Postfix config and then
>> offer to buy me lunch next time I'm in the city to persuade me to
>> make a patch to support this. :)
>>
>
> Are you a contributor to SA's development?
I've been known to commit enough good, and apparently not enough bad,
code to the SA code base.
> I guess the same compromise will exist in this scenario? Would it be
> difficult to get SA to see that the user is authenticated and just skip
> SPF but still do everything else?
In this case (SA knows the user is authenticated), SA does The Right
Thing (tm). All regex based tests and network tests against body
content are done and the appropriate network based checks are done
against the appropriate hosts.
> I see from the header in the message you sent that you have deployed
> DKIM. I'm hoping to do that as well but not for a while yet. Do similar
> problems arise with DKIM and SA as we've discussed here with SPF?
DKIM doesn't rely on any defined set of relays. It uses the envelope
sender (usually just the domain) and the signature found in the headers.
Also note that SPF isn't the only thing suffering from your trust path
issues, it's just a symptom you've noticed. You'll also currently be
doing dynablock checks against users you'd rather not be, along with a
whole slew of other tests that will FP when SA thinks it's testing mail
from some random system/zombie and not an authenticated user.
Let me know if you're running Postfix 2.3 and can enable the auth
headers in your config. I'll probably get to making a patch tonight as
long as the rain doesn't stop and I don't get distracted by the big
stash of fireworks I've accumulated. :)
Daryl
Re: Spamassassin and SPF
Posted by Gino Cerullo <gc...@pixelpointstudios.com>.
On 10-Jul-06, at 7:39 PM, Daryl C. W. O'Shea wrote:
> Gino Cerullo wrote:
>> On 10-Jul-06, at 5:49 PM, Daryl C. W. O'Shea wrote:
>>> If the MSA in question is *ONLY* an MSA, you're easiest quickest
>>> fix is to just not trust it (or mark it as trusted but not
>>> internal).
>> If I'm understanding you correctly, I'm fairly new to this, then
>> no, the MSA (Mail Submission Agent) is also the MTA (Mail Transfer
>> Agent.) All mail for the domains in question are received and
>> delivered by this one server.
>> trusted_networks lists my public and private IP addresses (WAN & LAN)
>> internal_networks lists only private IP addresses (LAN)
>
> These are incorrect for your setup. Trusted and internal networks
> should both list YOUR public and private IP addresses. IP
> addresses of other people could go in trusted networks only, but
> don't even bother with that.
>
> The only 'one' of your addresses that wouldn't go in internal
> networks would be your MSA if it was only an MSA, but yours isn't
> as you've said it's also working as an MX.
>
>
>> I think that's how those are suppose to work. I haven't bothered
>> to try and list the IPs where outside users may connect from since
>> that can be anywhere.
>
> So... fixing the above will resolve your SPF (and other) issues for
> LAN users. Add all the IP space you control to trusted networks
> (including your internal RFC 1938 ranges). Don't even bother with
> setting internal networks. In your case they'll be the same as
> your trusted networks, so SA will just copy them for you if you
> leave internal networks unset.
>
Thanks for the clarification. I'll make the changes.
>
> That still leaves a problem with your roaming users not being
> trusted. Postfix makes this oh so fun.
>
>
>>> If SA is running on that host, or it's also doing another mail
>>> function, MX or intermediate relay, etc., then describe your mail
>>> topology and we'll help you out.
>> Okay, this one server is running Postfix, Amavisd-new,
>> Spamassassin, ClamAV, pretty standard stuff. This server does not
>> relay to any other servers for mail delivery and is not a relay
>> for any servers. Both incoming and outgoing mail is handled by
>> Postfix which hands it off to Amavisd-new which calls Spamassassin
>> and ClamAV to scan the message. Clean email is handed back to
>> Postfix to complete delivery. Bad email is quarantined by Amavis-
>> new. At least that's my understanding of how it works.
>> Generally, because of SPF, all users submit directly to this
>> server and all outgoing email for the domains handled by this
>> server are delivered directly by this server, no relays involved.
>> Fairly simple stuff and it all works as expected except for SA.
>
>> This is what I'm seeing if it helps.
>> 0.9 SPF_FAIL SPF: sender does not match SPF record
>> (fail)
>> [SPF failed: Please see http://www.openspf.org/why.html?
>> sender=dmaguire%
>> 40maguiremarketing.com&ip=24.42.90.104&receiver=server.pixelpointstud
>> ios.lan] maguiremarketing.com is hosted by the server in question.
>> 24.42.90.104 is the IP address that he is connecting from. In this
>> case it's the IP assigned dynamically to his router by his cable
>> company (ISP).
>> Like I said the Postfix policy daemon that checks SPF correctly
>> ignores this IP address as it represents the MUA (Mail User
>> Agent.) I guess I'm expecting SA to know that as well but I guess
>> it doesn't.
>
> You can't expect SA to do something based on information that
> Postfix has and SA doesn't (because Postfix doesn't give it up, at
> least not easily), that is, SA doesn't know the user is an
> authenticated user while Postfix does.
>
This is what I thought. Thanks for confirming this as well.
>
> You have two options:
>
> 1) Quoting mouss...
>
> mouss wrote:
> > martin f krafft a écrit :
> >> Well, sure, this makes sense, but how can I support this standard
> >> use-case? Postfix adding a SASL-header that causes Spamassassin
> then
> >> to ignore the message isn't the solution as spammers would
> simply do
> >> that sooner or later. Short of whitelisting people, what should
> >> I do?
> >>
> >
> > how do you integrate SA with postfix?
> >
> > If using content_filter, then you could skip SA for authenticated
> users.
> > for instance:
> >
> > content_filter=
> > #or this to still check for viruses:
> > #content_filter=scan:[clamsmtp.domain.example]:10020
> > smtpd_recipient_restrictions =
> > ...
> > permit_sasl_authenticated
> > permit_mynetworks
> > reject_unauth_destination
> > check_client_access pcre:/etc/postfix/default_filter.pcre
> > ...
> >
> > == default_filter.pcre:
> > /./ FILTER scan:[amavisd.domain.example]:10024
> >
> > ('scan' is the name of the transport as defined in master.cf).
>
I see but is the trade-off here that if SA skips authenticated users
and they are compromised then they can become spam sources that
wouldn't be caught locally or does it only skip SPF and still do all
other scans?
>
> or
>
> 2) Upgrade to Postfix 2.3, if necessary, set
> "smtpd_sasl_authenticated_header yes" in your Postfix config and
> then offer to buy me lunch next time I'm in the city to persuade me
> to make a patch to support this. :)
>
Are you a contributor to SA's development?
I guess the same compromise will exist in this scenario? Would it be
difficult to get SA to see that the user is authenticated and just
skip SPF but still do everything else?
I see from the header in the message you sent that you have deployed
DKIM. I'm hoping to do that as well but not for a while yet. Do
similar problems arise with DKIM and SA as we've discussed here with
SPF?
No problem. I eat lunch...usually...if I remember to. It's amazing
how fast the day goes by when you're busy.
>
> Related info:
>
> http://www.postfix.org/postconf.5.html#smtpd_sasl_authenticated_header
> http://wiki.apache.org/spamassassin/DynablockIssues
>
>
> Daryl
>
Thanks for all your help and suggestions.
--
Gino Cerullo
Pixel Point Studios
21 Chesham Drive
Toronto, ON M3M 1W6
T: 416-247-7740
F: 416-247-7503
Re: Spamassassin and SPF
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
Daryl C. W. O'Shea wrote:
> So... fixing the above will resolve your SPF (and other) issues for LAN
> users. Add all the IP space you control to trusted networks (including
> your internal RFC 1938 ranges).
I just woke up and said, why'd I write that! I of course meant RFC 1918
addresses. I did send a birthday card for someone's 68th birthday
yesterday, maybe that's it.
Daryl
Re: Spamassassin and SPF
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
Gino Cerullo wrote:
>
> On 10-Jul-06, at 5:49 PM, Daryl C. W. O'Shea wrote:
>
>> If the MSA in question is *ONLY* an MSA, you're easiest quickest fix
>> is to just not trust it (or mark it as trusted but not internal).
>
> If I'm understanding you correctly, I'm fairly new to this, then no, the
> MSA (Mail Submission Agent) is also the MTA (Mail Transfer Agent.) All
> mail for the domains in question are received and delivered by this one
> server.
>
> trusted_networks lists my public and private IP addresses (WAN & LAN)
> internal_networks lists only private IP addresses (LAN)
These are incorrect for your setup. Trusted and internal networks
should both list YOUR public and private IP addresses. IP addresses of
other people could go in trusted networks only, but don't even bother
with that.
The only 'one' of your addresses that wouldn't go in internal networks
would be your MSA if it was only an MSA, but yours isn't as you've said
it's also working as an MX.
> I think that's how those are suppose to work. I haven't bothered to try
> and list the IPs where outside users may connect from since that can be
> anywhere.
So... fixing the above will resolve your SPF (and other) issues for LAN
users. Add all the IP space you control to trusted networks (including
your internal RFC 1938 ranges). Don't even bother with setting internal
networks. In your case they'll be the same as your trusted networks, so
SA will just copy them for you if you leave internal networks unset.
That still leaves a problem with your roaming users not being trusted.
Postfix makes this oh so fun.
>> If SA is running on that host, or it's also doing another mail
>> function, MX or intermediate relay, etc., then describe your mail
>> topology and we'll help you out.
>
> Okay, this one server is running Postfix, Amavisd-new, Spamassassin,
> ClamAV, pretty standard stuff. This server does not relay to any other
> servers for mail delivery and is not a relay for any servers. Both
> incoming and outgoing mail is handled by Postfix which hands it off to
> Amavisd-new which calls Spamassassin and ClamAV to scan the message.
> Clean email is handed back to Postfix to complete delivery. Bad email is
> quarantined by Amavis-new. At least that's my understanding of how it
> works.
>
> Generally, because of SPF, all users submit directly to this server and
> all outgoing email for the domains handled by this server are delivered
> directly by this server, no relays involved. Fairly simple stuff and it
> all works as expected except for SA.
> This is what I'm seeing if it helps.
>
> 0.9 SPF_FAIL SPF: sender does not match SPF record (fail)
> [SPF failed: Please see
> http://www.openspf.org/why.html?sender=dmaguire%40maguiremarketing.com&ip=24.42.90.104&receiver=server.pixelpointstudios.lan]
>
>
> maguiremarketing.com is hosted by the server in question.
>
> 24.42.90.104 is the IP address that he is connecting from. In this case
> it's the IP assigned dynamically to his router by his cable company (ISP).
>
> Like I said the Postfix policy daemon that checks SPF correctly ignores
> this IP address as it represents the MUA (Mail User Agent.) I guess I'm
> expecting SA to know that as well but I guess it doesn't.
You can't expect SA to do something based on information that Postfix
has and SA doesn't (because Postfix doesn't give it up, at least not
easily), that is, SA doesn't know the user is an authenticated user
while Postfix does.
You have two options:
1) Quoting mouss...
mouss wrote:
> martin f krafft a écrit :
>> Well, sure, this makes sense, but how can I support this standard
>> use-case? Postfix adding a SASL-header that causes Spamassassin then
>> to ignore the message isn't the solution as spammers would simply do
>> that sooner or later. Short of whitelisting people, what should
>> I do?
>>
>
> how do you integrate SA with postfix?
>
> If using content_filter, then you could skip SA for authenticated users.
> for instance:
>
> content_filter=
> #or this to still check for viruses:
> #content_filter=scan:[clamsmtp.domain.example]:10020
> smtpd_recipient_restrictions =
> ...
> permit_sasl_authenticated
> permit_mynetworks
> reject_unauth_destination
> check_client_access pcre:/etc/postfix/default_filter.pcre
> ...
>
> == default_filter.pcre:
> /./ FILTER scan:[amavisd.domain.example]:10024
>
> ('scan' is the name of the transport as defined in master.cf).
or
2) Upgrade to Postfix 2.3, if necessary, set
"smtpd_sasl_authenticated_header yes" in your Postfix config and then
offer to buy me lunch next time I'm in the city to persuade me to make a
patch to support this. :)
Related info:
http://www.postfix.org/postconf.5.html#smtpd_sasl_authenticated_header
http://wiki.apache.org/spamassassin/DynablockIssues
Daryl
Re: Spamassassin and SPF
Posted by Gino Cerullo <gc...@pixelpointstudios.com>.
On 10-Jul-06, at 5:49 PM, Daryl C. W. O'Shea wrote:
> If the MSA in question is *ONLY* an MSA, you're easiest quickest
> fix is to just not trust it (or mark it as trusted but not internal).
If I'm understanding you correctly, I'm fairly new to this, then no,
the MSA (Mail Submission Agent) is also the MTA (Mail Transfer
Agent.) All mail for the domains in question are received and
delivered by this one server.
trusted_networks lists my public and private IP addresses (WAN & LAN)
internal_networks lists only private IP addresses (LAN)
I think that's how those are suppose to work. I haven't bothered to
try and list the IPs where outside users may connect from since that
can be anywhere.
>
> If SA is running on that host, or it's also doing another mail
> function, MX or intermediate relay, etc., then describe your mail
> topology and we'll help you out.
Okay, this one server is running Postfix, Amavisd-new, Spamassassin,
ClamAV, pretty standard stuff. This server does not relay to any
other servers for mail delivery and is not a relay for any servers.
Both incoming and outgoing mail is handled by Postfix which hands it
off to Amavisd-new which calls Spamassassin and ClamAV to scan the
message. Clean email is handed back to Postfix to complete delivery.
Bad email is quarantined by Amavis-new. At least that's my
understanding of how it works.
Generally, because of SPF, all users submit directly to this server
and all outgoing email for the domains handled by this server are
delivered directly by this server, no relays involved. Fairly simple
stuff and it all works as expected except for SA.
>
> Daryl
>
This is what I'm seeing if it helps.
0.9 SPF_FAIL SPF: sender does not match SPF record
(fail)
[SPF failed: Please see http://www.openspf.org/why.html?
sender=dmaguire%
40maguiremarketing.com&ip=24.42.90.104&receiver=server.pixelpointstudios
.lan]
maguiremarketing.com is hosted by the server in question.
24.42.90.104 is the IP address that he is connecting from. In this
case it's the IP assigned dynamically to his router by his cable
company (ISP).
Like I said the Postfix policy daemon that checks SPF correctly
ignores this IP address as it represents the MUA (Mail User Agent.) I
guess I'm expecting SA to know that as well but I guess it doesn't.
--
Gino Cerullo
Pixel Point Studios
21 Chesham Drive
Toronto, ON M3M 1W6
T: 416-247-7740
F: 416-247-7503
Re: Spamassassin and SPF
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
If the MSA in question is *ONLY* an MSA, you're easiest quickest fix is
to just not trust it (or mark it as trusted but not internal).
If SA is running on that host, or it's also doing another mail function,
MX or intermediate relay, etc., then describe your mail topology and
we'll help you out.
Daryl