You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Flemming Jacobsen <fj...@batmule.dk> on 2012/06/19 22:21:39 UTC
USER_IN_WHITELIST and SPF_FAIL
Hey
I finally got around to enabling SPF checks in SA.
(v. 3.3.2, via spamd on FreeBSD)
It appears that even though SPF checks fail (i.e. SPF_FAIL),
USER_IN_WHITELIST still adds -100 points to the score.
Since the sender probably is spoofed, should USER_IN_WHITELIST
not be ignored/neutral (not sure of the terminology here)?
Regards,
Flemming Jacobsen
--
Flemming Jacobsen Email: fj@batmule.dk
"It is hard to believe that a man is telling the truth when you know that
you would lie if you were in his place." -- H. L. Mencken
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by Benny Pedersen <me...@junc.org>.
Den 2012-06-19 22:21, Flemming Jacobsen skrev:
> It appears that even though SPF checks fail (i.e. SPF_FAIL),
> USER_IN_WHITELIST still adds -100 points to the score.
> Since the sender probably is spoofed, should USER_IN_WHITELIST
> not be ignored/neutral (not sure of the terminology here)?
nope, whitelist_from is stupid in the first place
but since so many use it, it will properly stay forever :(
imidiate fix is to:
score USER_IN_WHITELIST -0.01
or
meta spf fails with user_in_* (insecure)
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by RW <rw...@googlemail.com>.
On Wed, 20 Jun 2012 18:38:49 +0200
Flemming Jacobsen wrote:
> RW wrote:
> > On Wed, 20 Jun 2012 11:33:49 +0200 Per Jessen wrote:
> > > RW wrote:
> > > > What I mean is that if I whitelist a private email address, the
> > > > chances of a spammer ever sending me a spam spoofing that
> > > > address is very small.
> > >
> > > Happened to me twice only yesterday - somebody sent me mails
> > > appearing to come from one of my email addresses. I don't think
> > > it's as rare an event as you suggest.
> >
> > Are you being deliberately obtuse? Of course that happens all the
> > time, but why would one whitelist such an address?
>
> Because you use email to send yourself reminder notes or small
> files. I have addresses on several distinct systems (private,
> work, google, user group, ...).
> And I whitelist them because I do not want mail to get lost.
If it's an unrelated external address then it's just one address in
billions and it wont be randomly spoofed.
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by Benny Pedersen <me...@junc.org>.
Den 2012-06-20 18:38, Flemming Jacobsen skrev:
> Because you use email to send yourself reminder notes or small
> files. I have addresses on several distinct systems (private,
> work, google, user group, ...).
> And I whitelist them because I do not want mail to get lost.
with shared imap folders nothing get lost, all that mail does not need
to travel, but implementions need to be more usefull, its like forwards
that breaks spf, its lie, since known forward hosts must be
trusted_networks, draw back in make 0.0.0.0/0 trusted_networks it
removes all domain based trustness
for such problems it would be more usefull to disable dnsrbl and only
check content based on body/rawbody
unless one use rbl in mta
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by Flemming Jacobsen <fj...@batmule.dk>.
RW wrote:
> On Wed, 20 Jun 2012 11:33:49 +0200 Per Jessen wrote:
> > RW wrote:
> > > What I mean is that if I whitelist a private email address, the
> > > chances of a spammer ever sending me a spam spoofing that address is
> > > very small.
> >
> > Happened to me twice only yesterday - somebody sent me mails appearing
> > to come from one of my email addresses. I don't think it's as rare an
> > event as you suggest.
>
> Are you being deliberately obtuse? Of course that happens all the
> time, but why would one whitelist such an address?
Because you use email to send yourself reminder notes or small
files. I have addresses on several distinct systems (private,
work, google, user group, ...).
And I whitelist them because I do not want mail to get lost.
Regards,
Flemming
--
Flemming Jacobsen Email: fj@batmule.dk
"There is nobody so irritating as somebody with less intelligence
and more sense than we have." -- Don Herold
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by RW <rw...@googlemail.com>.
On Wed, 20 Jun 2012 11:33:49 +0200
Per Jessen wrote:
> RW wrote:
>
> > On Wed, 20 Jun 2012 03:25:53 +0200
> > Benny Pedersen wrote:
> >
> >> Den 2012-06-20 03:09, RW skrev:
> >>
> >> > The overwhelming majority of email addresses are never spoofed.
> >
> >> seen from my mta logs off sender addresses that miss the smtp auth
> >> password here postfix dont agree with you, if sender uses something
> >> belongs to my domain i may start asking for passwords, this check
> >> is not needing spf or dkim or even dmarc tests
> >
> > I've no idea what that means, but what I wrote wasn't entirely
> > clear - particularly when taken out context.
> >
> > What I mean is that if I whitelist a private email address, the
> > chances of a spammer ever sending me a spam spoofing that address is
> > very small.
>
> Happened to me twice only yesterday - somebody sent me mails appearing
> to come from one of my email addresses. I don't think it's as rare an
> event as you suggest.
Are you being deliberately obtuse? Of course that happens all the
time, but why would one whitelist such an address?
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by Per Jessen <pe...@computer.org>.
RW wrote:
> On Wed, 20 Jun 2012 03:25:53 +0200
> Benny Pedersen wrote:
>
>> Den 2012-06-20 03:09, RW skrev:
>>
>> > The overwhelming majority of email addresses are never spoofed.
>
>> seen from my mta logs off sender addresses that miss the smtp auth
>> password here postfix dont agree with you, if sender uses something
>> belongs to my domain i may start asking for passwords, this check is
>> not needing spf or dkim or even dmarc tests
>
> I've no idea what that means, but what I wrote wasn't entirely clear -
> particularly when taken out context.
>
> What I mean is that if I whitelist a private email address, the
> chances of a spammer ever sending me a spam spoofing that address is
> very small.
Happened to me twice only yesterday - somebody sent me mails appearing
to come from one of my email addresses. I don't think it's as rare an
event as you suggest.
--
Per Jessen, Zürich (23.2°C)
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by RW <rw...@googlemail.com>.
On Wed, 20 Jun 2012 03:25:53 +0200
Benny Pedersen wrote:
> Den 2012-06-20 03:09, RW skrev:
>
> > The overwhelming majority of email addresses are never spoofed.
> seen from my mta logs off sender addresses that miss the smtp auth
> password here postfix dont agree with you, if sender uses something
> belongs to my domain i may start asking for passwords, this check is
> not needing spf or dkim or even dmarc tests
I've no idea what that means, but what I wrote wasn't entirely clear -
particularly when taken out context.
What I mean is that if I whitelist a private email address, the chances
of a spammer ever sending me a spam spoofing that address is very
small.
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by Benny Pedersen <me...@junc.org>.
Den 2012-06-20 03:09, RW skrev:
> The overwhelming majority of email addresses are never spoofed.
seen from my mta logs off sender addresses that miss the smtp auth
password here postfix dont agree with you, if sender uses something
belongs to my domain i may start asking for passwords, this check is not
needing spf or dkim or even dmarc tests
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by RW <rw...@googlemail.com>.
On Wed, 20 Jun 2012 11:22:08 +0200
Per Jessen wrote:
> RW wrote:
> > Not if someone sends an email through a different mail system,
>
> I think that is what "whitelist_allows_relays" is intended to take
> care of.
If it made a difference to the case I was referring to then it would
effectively turn whitelist_from_rcvd into whitelist_from for
the specified addresses.
I looked it up, "whitelist_allows_relays" is a list of addresses
excluded from check_forged_in_whitelist, which is not used in the
current rules.
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by Per Jessen <pe...@computer.org>.
RW wrote:
> On Tue, 19 Jun 2012 19:14:11 -0400
> Jeff Mincy wrote:
>
>> From: RW <rw...@googlemail.com>
>> Date: Tue, 19 Jun 2012 23:43:57 +0100
>
>> If used sensibly USER_IN_WHITELIST is probably the most reliable
>> rule we have, for the overwhelming majority of addresses it's far
>> more accurate than spf based whitelisting. It's not always right to
>> treat users as idiots.
>>
>> Huh? What you mean by used sensibly?
>
> I mean, don't use it on well-known addresses, or if you're a candidate
> for spear-phishing and can't be trusted not to fall for it. Don't
> whitelist domains unless they are extremely obscure.
>
>> whitelist_from_rcvd is very reliable.
>
> Not if someone sends an email through a different mail system,
I think that is what "whitelist_allows_relays" is intended to take care
of.
--
Per Jessen, Zürich (23.2°C)
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by RW <rw...@googlemail.com>.
On Tue, 19 Jun 2012 19:14:11 -0400
Jeff Mincy wrote:
> From: RW <rw...@googlemail.com>
> Date: Tue, 19 Jun 2012 23:43:57 +0100
> If used sensibly USER_IN_WHITELIST is probably the most reliable
> rule we have, for the overwhelming majority of addresses it's far
> more accurate than spf based whitelisting. It's not always right to
> treat users as idiots.
>
> Huh? What you mean by used sensibly?
I mean, don't use it on well-known addresses, or if you're a candidate
for spear-phishing and can't be trusted not to fall for it. Don't
whitelist domains unless they are extremely obscure.
> whitelist_from_rcvd is very reliable.
Not if someone sends an email through a different mail system, which is
a scenario where Bayes is much more likely to miss-classify and an FP
is most likely. It's also broken by forwarding, like spf is.
> whitelist_from is trivial to spoof.
The overwhelming majority of email addresses are never spoofed.
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by Jeff Mincy <je...@delphioutpost.com>.
From: RW <rw...@googlemail.com>
Date: Tue, 19 Jun 2012 23:43:57 +0100
On Tue, 19 Jun 2012 18:02:28 -0400
Jeff Mincy wrote:
> From: John Hardin <jh...@impsec.org>
> Date: Tue, 19 Jun 2012 14:44:29 -0700 (PDT)
>
> On Tue, 19 Jun 2012, Benny Pedersen wrote:
>
> > Den 2012-06-19 22:39, Kevin A. McGrail skrev:
> >
> >> I think that's the concept behind the whitelist_from_spf
> >
> > but some use whitelist_from, its nothing new there :=)
> >
> > can user_in_whitelist be changed to not have -100 as default
> > score, or is whitelist_from planned for removements ?
>
> It's needed for whan none of the other more-strict whitelist
> options will work, so we can't get just rid of it.
>
> True.
>
> I'd suggest instead a lint warning if it is used, alerting the
> admin that it's discouraged and that it has problems like this and is
> very easy to spoof.
>
> How about creating a different score for whitelist_from that is
> separate from whitelist_from_rcvd? For example, whitelist_from could
> trigger USER_IN_SIMPLE_WHITELIST (or some other variation). The
> description of the test could include warnings about how easy
> it is to spoof whitelist_from.
If used sensibly USER_IN_WHITELIST is probably the most reliable rule we
have, for the overwhelming majority of addresses it's far more accurate
than spf based whitelisting. It's not always right to treat users as
idiots.
Huh? What you mean by used sensibly? whitelist_from_rcvd is very
reliable. whitelist_from is trivial to spoof. whitelist_from_rcvd
and whitelist_from both trigger USER_IN_WHITELIST.
It is easy to get into trouble using whitelist_from - having a
separate score just for whitelist_from would make identifying the
problem easier for the user.
-jeff
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by RW <rw...@googlemail.com>.
On Tue, 19 Jun 2012 18:02:28 -0400
Jeff Mincy wrote:
> From: John Hardin <jh...@impsec.org>
> Date: Tue, 19 Jun 2012 14:44:29 -0700 (PDT)
>
> On Tue, 19 Jun 2012, Benny Pedersen wrote:
>
> > Den 2012-06-19 22:39, Kevin A. McGrail skrev:
> >
> >> I think that's the concept behind the whitelist_from_spf
> >
> > but some use whitelist_from, its nothing new there :=)
> >
> > can user_in_whitelist be changed to not have -100 as default
> > score, or is whitelist_from planned for removements ?
>
> It's needed for whan none of the other more-strict whitelist
> options will work, so we can't get just rid of it.
>
> True.
>
> I'd suggest instead a lint warning if it is used, alerting the
> admin that it's discouraged and that it has problems like this and is
> very easy to spoof.
>
> How about creating a different score for whitelist_from that is
> separate from whitelist_from_rcvd? For example, whitelist_from could
> trigger USER_IN_SIMPLE_WHITELIST (or some other variation). The
> description of the test could include warnings about how easy
> it is to spoof whitelist_from.
If used sensibly USER_IN_WHITELIST is probably the most reliable rule we
have, for the overwhelming majority of addresses it's far more accurate
than spf based whitelisting. It's not always right to treat users as
idiots.
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by Benny Pedersen <me...@junc.org>.
Den 2012-06-20 14:05, Greg Troxel skrev:
> That way I could do:
>
> whitelist_from -5 foo@yahoo.com
AWL plugin basicly could be extended to use dkim/spf and more bound to
whitelist_* so the awl score is more live calculated, with default awl
its bound to 0.0.x.x/16 but it could be changed to /8 /24 /32 matching,
so scores is more accurate pr sender
but your way could very well extend problems or usefullness depending
on with side of the screen one sits :)
awl can track dkim senders, but it would be nice dkim is not alone
there
awl is imho dropped in spamassassin 3.4 and replaced with history
plugin, i dont know what or why or even code to this plugin
maybe score sets should just be extended to more then 4 colums ?
score foo set1 set2 set3 set4 spf dkim
just an stupid idear maybe ?
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 6/20/2012 8:05 AM, Greg Troxel wrote:
> I would like to see...
As an open source project, we encourage people to submit patches and
step up to coding on the project.
You can really start small with one line patches and I'll do my best to
support you.
Regards,
KAM
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by Greg Troxel <gd...@ir.bbn.com>.
My suggestion was intended to minimize the effect on existing
behavior. I agree, it would probably be a very good idea to allow
whitelist_from to be scored differently than the other whitelist
variants, and to ship it with a smaller default score, but that change
is fairly disruptive.
I would like to see
whitelist_score_from points address
which acts just like "whitelist_from address", but which has a score of
"points" rather than some fixed score.
That way I could do:
whitelist_from -5 foo@yahoo.com
for people that post legit but spammy-looking mail to mailinglists, and
get their regular mail in the right folder instead of a spam folder, but
not let their account-hijacked spam bleed through like -100 would do.
And also use -20 for people I know, -50 for customers, etc.
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by John Hardin <jh...@impsec.org>.
On Tue, 19 Jun 2012, Jeff Mincy wrote:
> From: John Hardin <jh...@impsec.org>
>
> I'd suggest instead a lint warning if it is used, alerting the admin that
> it's discouraged and that it has problems like this and is very easy to
> spoof.
>
> How about creating a different score for whitelist_from that is
> separate from whitelist_from_rcvd? For example, whitelist_from could
> trigger USER_IN_SIMPLE_WHITELIST (or some other variation). The
> description of the test could include warnings about how easy
> it is to spoof whitelist_from.
My suggestion was intended to minimize the effect on existing behavior. I
agree, it would probably be a very good idea to allow whitelist_from to be
scored differently than the other whitelist variants, and to ship it with
a smaller default score, but that change is fairly disruptive.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Usually Microsoft doesn't develop products, we buy products.
-- Arno Edelmann, Microsoft product manager
-----------------------------------------------------------------------
15 days until the 236th anniversary of the Declaration of Independence
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by Jeff Mincy <je...@delphioutpost.com>.
From: John Hardin <jh...@impsec.org>
Date: Tue, 19 Jun 2012 14:44:29 -0700 (PDT)
On Tue, 19 Jun 2012, Benny Pedersen wrote:
> Den 2012-06-19 22:39, Kevin A. McGrail skrev:
>
>> I think that's the concept behind the whitelist_from_spf
>
> but some use whitelist_from, its nothing new there :=)
>
> can user_in_whitelist be changed to not have -100 as default score, or is
> whitelist_from planned for removements ?
It's needed for whan none of the other more-strict whitelist options will
work, so we can't get just rid of it.
True.
I'd suggest instead a lint warning if it is used, alerting the admin that
it's discouraged and that it has problems like this and is very easy to
spoof.
How about creating a different score for whitelist_from that is
separate from whitelist_from_rcvd? For example, whitelist_from could
trigger USER_IN_SIMPLE_WHITELIST (or some other variation). The
description of the test could include warnings about how easy
it is to spoof whitelist_from.
-jeff
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by Per Jessen <pe...@computer.org>.
John Hardin wrote:
> On Tue, 19 Jun 2012, Benny Pedersen wrote:
>
>> Den 2012-06-19 22:39, Kevin A. McGrail skrev:
>>
>>> I think that's the concept behind the whitelist_from_spf
>>
>> but some use whitelist_from, its nothing new there :=)
>>
>> can user_in_whitelist be changed to not have -100 as default score,
>> or is whitelist_from planned for removements ?
>
> It's needed for whan none of the other more-strict whitelist options
> will work, so we can't get just rid of it.
>
> I'd suggest instead a lint warning if it is used, alerting the admin
> that it's discouraged and that it has problems like this and is very
> easy to spoof.
It's well documented. From the man page:
whitelist_from add@ress.com
Used to whitelist sender addresses which send mail that is often tagged
(incorrectly) as spam.
Use of this setting is not recommended, since it blindly trusts the
message, which is routinely and easily forged by spammers and phish
senders. The recommended solution is to instead use "whitelist_auth" or
other authenticated whitelisting methods, or "whitelist_from_rcvd".
--
Per Jessen, Zürich (21.1°C)
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by Benny Pedersen <me...@junc.org>.
Den 2012-06-19 23:44, John Hardin skrev:
> I'd suggest instead a lint warning if it is used, alerting the admin
> that it's discouraged and that it has problems like this and is very
> easy to spoof.
fair, but Flemming might choise some meta like this:
meta WHITELIST_INSECURE_SPF (USER_IN_WHITELIST && SPF_FAIL)
score WHITELIST_INSECURE_SPF 50
but since Flemming did not provide an sample there might be other
options, eg why accept spf_fail in mta ?
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by John Hardin <jh...@impsec.org>.
On Tue, 19 Jun 2012, Benny Pedersen wrote:
> Den 2012-06-19 22:39, Kevin A. McGrail skrev:
>
>> I think that's the concept behind the whitelist_from_spf
>
> but some use whitelist_from, its nothing new there :=)
>
> can user_in_whitelist be changed to not have -100 as default score, or is
> whitelist_from planned for removements ?
It's needed for whan none of the other more-strict whitelist options will
work, so we can't get just rid of it.
I'd suggest instead a lint warning if it is used, alerting the admin that
it's discouraged and that it has problems like this and is very easy to
spoof.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Liberals love sex ed because it teaches kids to be safe around their
sex organs. Conservatives love gun education because it teaches kids
to be safe around guns. However, both believe that the other's
education goals lead to dangers too terrible to contemplate.
-----------------------------------------------------------------------
15 days until the 236th anniversary of the Declaration of Independence
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by Axb <ax...@gmail.com>.
On 06/19/2012 11:34 PM, Benny Pedersen wrote:
> Den 2012-06-19 22:39, Kevin A. McGrail skrev:
>
>> I think that's the concept behind the whitelist_from_spf
>
> but some use whitelist_from, its nothing new there :=)
>
> can user_in_whitelist be changed to not have -100 as default score, or
> is whitelist_from planned for removements ?
no & no
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by Benny Pedersen <me...@junc.org>.
Den 2012-06-19 22:39, Kevin A. McGrail skrev:
> I think that's the concept behind the whitelist_from_spf
but some use whitelist_from, its nothing new there :=)
can user_in_whitelist be changed to not have -100 as default score, or
is whitelist_from planned for removements ?
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 6/19/2012 4:21 PM, Flemming Jacobsen wrote:
> Hey
>
> I finally got around to enabling SPF checks in SA.
> (v. 3.3.2, via spamd on FreeBSD)
>
> It appears that even though SPF checks fail (i.e. SPF_FAIL),
> USER_IN_WHITELIST still adds -100 points to the score.
> Since the sender probably is spoofed, should USER_IN_WHITELIST
> not be ignored/neutral (not sure of the terminology here)?
>
I think that's the concept behind the whitelist_from_spf
http://spamassassin.apache.org/full/3.3.x/doc/Mail_SpamAssassin_Plugin_SPF.html
Regards,
KAM
Re: USER_IN_WHITELIST and SPF_FAIL
Posted by John Hardin <jh...@impsec.org>.
On Tue, 19 Jun 2012, Flemming Jacobsen wrote:
> I finally got around to enabling SPF checks in SA.
> (v. 3.3.2, via spamd on FreeBSD)
>
> It appears that even though SPF checks fail (i.e. SPF_FAIL),
> USER_IN_WHITELIST still adds -100 points to the score.
> Since the sender probably is spoofed, should USER_IN_WHITELIST
> not be ignored/neutral (not sure of the terminology here)?
Which whitelist is the problematic user in? whitelist_from is a naive
check of the from address and is trivially easy to spoof.
You should review your whitelists and, now that you have SPF working, move
senders that are in authenticated domains from whitelist_from to
whitelist_auth so that you take advantage of SPF (and DKIM, if you have
that working as well).
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Usually Microsoft doesn't develop products, we buy products.
-- Arno Edelmann, Microsoft product manager
-----------------------------------------------------------------------
15 days until the 236th anniversary of the Declaration of Independence