You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Alex <my...@gmail.com> on 2010/10/05 16:40:07 UTC
Whitelist questions
Hi,
I have an email that I'm trying to whitelist using whitelist_from_rcvd
and it's not working as I expect. I've created an entry:
whitelist_from_rcvd user@lanyon.com savvis.net
Here is the corresponding received header:
X-Envelope-From: <us...@lanyon.com>
Received: from S253906HZ1EW06.usstls6-hosting.savvis.net (unknown
[209.16.192.170])
Is it because there is no reverse DNS entry?
Also, am I somehow not using the AWL correctly? It's actually adding
points and not subtracting them:
* 0.0 RELAYCOUNTRY_US Relayed through United States
* 0.3 LOC_RCVD_UNK Contains unknown IP in Received header
* 1.4 BOTNET Relay might be a spambot or virusbot
* [botnet0.8,ip=209.16.192.170,rdns=Lanyon.com,maildomain=lanyon.com,baddns]
* 0.8 RDNS_NONE Delivered to internal network by a host with no rDNS
* 1.0 FILL_THIS_FORM_LONG Fill in a form with personal information
* 1.0 FILL_THIS_FORM Fill in a form with personal information
* 1.5 FILL_THIS_FORM_FRAUD_PHISH Answer suspicious question(s)
* 0.7 AWL AWL: From: address is in the auto white-list
Under what circumstances would this happen?
Thanks,
Alex
Re: Whitelist questions
Posted by RW <rw...@googlemail.com>.
On Tue, 5 Oct 2010 10:40:07 -0400
Alex <my...@gmail.com> wrote:
> Hi,
>
> I have an email that I'm trying to whitelist using whitelist_from_rcvd
> and it's not working as I expect. I've created an entry:
>
>...
>
> Is it because there is no reverse DNS entry?
Yes. It would be nice to have the option look it up when it's missing,
but that's not supported.
> Also, am I somehow not using the AWL correctly? It's actually adding
> points and not subtracting them:
>
> ...
>
> Under what circumstances would this happen?
It moves the score towards the mean score for that sender/ip. The name
Autowhitelist is misleading, it's really a score averager.
Re: Whitelist questions
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Tue, 2010-10-05 at 13:16 -0700, John Hardin wrote:
> On Tue, 5 Oct 2010, Karsten Bräckelmann wrote:
Your MUA still can't handle UTF-8, eh? Fixed my name. ;)
> > If there really is no way to use whitelist_from_rcvd, you of course
> > always can write custom header rules, matching against the pseudo header
> > X-Spam-Relays-Internal or friends, carefully constructing the RE to
> > match a specific Received header by constraining it with the square
> > brackets surrounding each relay.
>
> Perhaps whitelist_from_rcvd should recoginze IP syntax and ignore the
> rDNS, so this would work:
spamassassin -D < $msg 2>&1 | grep X-Spam-Relay # untested
> whitelist_from_rcvd user@lanyon.com [209.16.192.170]
>
> ...not that I'd want to maintain IP-based whitelists...
It is quite easy to write such a white-listing rule yourself, using the
Relay pseudo-headers. And yes, a whitelist_from_ip setting plugin as
Alex mentioned would be quite easy as well.
However, if there is any need for a hack like that, it likely doesn't
affect a single recipient (which would rule out writing the plugin
anyway), but lots of recipients. Better fix the sender's infra, so
everyone will benefit from that.
Fix sender. Rather than having every single recipient, now and in the
future, fix it locally for him.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Whitelist questions
Posted by Yet Another Ninja <sa...@alexb.ch>.
On 2010-10-05 22:16, John Hardin wrote:
> On Tue, 5 Oct 2010, Karsten Br�ckelmann wrote:
>
>> If there really is no way to use whitelist_from_rcvd, you of course
>> always can write custom header rules, matching against the pseudo header
>> X-Spam-Relays-Internal or friends, carefully constructing the RE to
>> match a specific Received header by constraining it with the square
>> brackets surrounding each relay.
>
> Perhaps whitelist_from_rcvd should recoginze IP syntax and ignore the
> rDNS, so this would work:
>
> whitelist_from_rcvd user@lanyon.com [209.16.192.170]
>
> ....not that I'd want to maintain IP-based whitelists...
wasn't there an whitelist_fromip plugin floating around sometime ago?
Re: Whitelist questions
Posted by John Hardin <jh...@impsec.org>.
On Tue, 5 Oct 2010, Karsten Br�ckelmann wrote:
> If there really is no way to use whitelist_from_rcvd, you of course
> always can write custom header rules, matching against the pseudo header
> X-Spam-Relays-Internal or friends, carefully constructing the RE to
> match a specific Received header by constraining it with the square
> brackets surrounding each relay.
Perhaps whitelist_from_rcvd should recoginze IP syntax and ignore the
rDNS, so this would work:
whitelist_from_rcvd user@lanyon.com [209.16.192.170]
...not that I'd want to maintain IP-based whitelists...
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
When designing software, any time you think to yourself "a user
would never be stupid enough to do *that*", you're wrong.
-----------------------------------------------------------------------
73 days until TRON Legacy
Re: Whitelist questions
Posted by Alex <my...@gmail.com>.
Hi,
>> $ host 209.16.192.170
>> 170.192.16.209.in-addr.arpa domain name pointer Lanyon.com.
>
> but they don't match:
> host Lanyon.com
> Lanyon.com has address 97.74.177.132
>
> 97.74.177.132
> 132.177.74.97.in-addr.arpa domain name pointer
> ip-97-74-177-132.ip.secureserver.net.
Ah, right, I see.
> why not just use something like 'ob.lanyon.com', in your HELO, FQDN, and
> make sure that both FWD and RDNS match?
This is mail that I am receiving and trying to whitelist, not mail
that I am sending.
Thanks,
Alex
Re: Whitelist questions
Posted by Michael Scheidell <mi...@secnap.com>.
On 10/5/10 12:45 PM, Alex wrote:
> $ host 209.16.192.170
> 170.192.16.209.in-addr.arpa domain name pointer Lanyon.com.
but they don't match:
host Lanyon.com
Lanyon.com has address 97.74.177.132
97.74.177.132
132.177.74.97.in-addr.arpa domain name pointer
ip-97-74-177-132.ip.secureserver.net.
why not just use something like 'ob.lanyon.com', in your HELO, FQDN, and
make sure that both FWD and RDNS match?
--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
ISN: 1259*1300
> *| *SECNAP Network Security Corporation
* Certified SNORT Integrator
* 2008-9 Hot Company Award Winner, World Executive Alliance
* Five-Star Partner Program 2009, VARBusiness
* Best in Email Security,2010: Network Products Guide
* King of Spam Filters, SC Magazine 2008
______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________
Re: Whitelist questions
Posted by Alex <my...@gmail.com>.
Hi,
>> $ host S253906HZ1EW06.usstls6-hosting.savvis.net
>> Host S253906HZ1EW06.usstls6-hosting.savvis.net not found: 3(NXDOMAIN)
>
> Err, you're doing rDNS lookup for the connecting host's IP, not the
> rather arbitrary HELO as you just did.
Okay, understood. I'm able to resolve that IP, though:
$ host 209.16.192.170
170.192.16.209.in-addr.arpa domain name pointer Lanyon.com.
The postfix resolv.conf is the same as the one in /etc.
> If there really is no way to use whitelist_from_rcvd, you of course
> always can write custom header rules, matching against the pseudo header
Yes, thought of that too, but agree that is not an ideal solution.
Could there be another cause?
Thanks,
Alex
Re: Whitelist questions
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Tue, 2010-10-05 at 11:51 -0400, Alex wrote:
> > As the documentation [1] clearly states, the second value (a) is a
> > string matched against the relay's rDNS in the Received headers, and
> > (b) it is your MX's responsibility to perform the rDNS lookup and add it
> > to the header.
> > $ host 209.16.192.170
> > 170.192.16.209.in-addr.arpa domain name pointer Lanyon.com.
> >
> > So, fix your MX. :) With an rDNS entry in the header, you will need to
>
> I'm not it's a problem with my system. It's not that IP that has the
> problem, but with the host itself:
>
> $ host S253906HZ1EW06.usstls6-hosting.savvis.net
> Host S253906HZ1EW06.usstls6-hosting.savvis.net not found: 3(NXDOMAIN)
Err, you're doing rDNS lookup for the connecting host's IP, not the
rather arbitrary HELO as you just did.
> I suppose I could add that host to /etc/hosts, but is there another
> way to whitelist mail from this host/domain to a specific user? There
> are also no SPF records to use...
If there really is no way to use whitelist_from_rcvd, you of course
always can write custom header rules, matching against the pseudo header
X-Spam-Relays-Internal or friends, carefully constructing the RE to
match a specific Received header by constraining it with the square
brackets surrounding each relay.
However, I do not see yet why that should be necessary in your case.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Whitelist questions
Posted by Alex <my...@gmail.com>.
Hi,
>> X-Envelope-From: <us...@lanyon.com>
>> Received: from S253906HZ1EW06.usstls6-hosting.savvis.net (unknown
>> [209.16.192.170])
>>
>> Is it because there is no reverse DNS entry?
>
> As the documentation [1] clearly states, the second value (a) is a
> string matched against the relay's rDNS in the Received headers, and
> (b) it is your MX's responsibility to perform the rDNS lookup and add it
> to the header.
>
> The latter is key, because despite "no reverse DNS entry" being
> ambiguous, there actually is an rDNS entry for the IP. In DNS, it's
> merely missing from the Received header.
>
> $ host 209.16.192.170
> 170.192.16.209.in-addr.arpa domain name pointer Lanyon.com.
>
> So, fix your MX. :) With an rDNS entry in the header, you will need to
I'm not it's a problem with my system. It's not that IP that has the
problem, but with the host itself:
$ host S253906HZ1EW06.usstls6-hosting.savvis.net
Host S253906HZ1EW06.usstls6-hosting.savvis.net not found: 3(NXDOMAIN)
I suppose I could add that host to /etc/hosts, but is there another
way to whitelist mail from this host/domain to a specific user? There
are also no SPF records to use...
Thanks again,
Alex
Re: Whitelist questions
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Tue, 2010-10-05 at 10:40 -0400, Alex wrote:
> I have an email that I'm trying to whitelist using whitelist_from_rcvd
> and it's not working as I expect. I've created an entry:
>
> whitelist_from_rcvd user@lanyon.com savvis.net
>
> Here is the corresponding received header:
>
> X-Envelope-From: <us...@lanyon.com>
> Received: from S253906HZ1EW06.usstls6-hosting.savvis.net (unknown
> [209.16.192.170])
>
> Is it because there is no reverse DNS entry?
As the documentation [1] clearly states, the second value (a) is a
string matched against the relay's rDNS in the Received headers, and
(b) it is your MX's responsibility to perform the rDNS lookup and add it
to the header.
The latter is key, because despite "no reverse DNS entry" being
ambiguous, there actually is an rDNS entry for the IP. In DNS, it's
merely missing from the Received header.
$ host 209.16.192.170
170.192.16.209.in-addr.arpa domain name pointer Lanyon.com.
So, fix your MX. :) With an rDNS entry in the header, you will need to
adjust your whitelist setting to use the actual rDNS, rather than the
savvis.net HELO mystery.
[1] http://spamassassin.apache.org/full/3.2.x/doc/Mail_SpamAssassin_Conf.html#whitelist_and_blacklist_options
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Whitelist questions
Posted by Joseph Brennan <br...@columbia.edu>.
David B Funk <db...@engineering.uiowa.edu> wrote:
>> Notice also that the rule checks the header From:, not the envelope,
>> and they could be different.
>
> When did that change?
Sorry. I am wrong.
Joseph Brennan
Columbia University Information Technology
Re: Whitelist questions
Posted by David B Funk <db...@engineering.uiowa.edu>.
On Tue, 5 Oct 2010, Joseph Brennan wrote:
>
> --On Tuesday, October 5, 2010 10:40 -0400 Alex <my...@gmail.com>
> wrote:
>
> > I have an email that I'm trying to whitelist using whitelist_from_rcvd
> > and it's not working as I expect. I've created an entry:
> >
[snip..]
>
> Notice also that the rule checks the header From:, not the envelope,
> and they could be different.
When did that change?
Quoting from the docs for Mail::SpamAssassin::Conf it used to be:
The headers checked for whitelist addresses are as follows: if "Resent-From" is set, use that; otherwise check all
addresses taken from the following set of headers:
Envelope-Sender
Resent-Sender
X-Envelope-From
From
In addition, the "envelope sender" data, taken from the SMTP envelope data where this is available, is looked up.
I distinctly remember setting up def_whitelist_from_rcvd entries to work
with envelope from addresses for "listwashing".
--
Dave Funk University of Iowa
<dbfunk (at) engineering.uiowa.edu> College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{
Re: Whitelist questions
Posted by Joseph Brennan <br...@columbia.edu>.
--On Tuesday, October 5, 2010 10:40 -0400 Alex <my...@gmail.com>
wrote:
> I have an email that I'm trying to whitelist using whitelist_from_rcvd
> and it's not working as I expect. I've created an entry:
>
> whitelist_from_rcvd user@lanyon.com savvis.net
>
> Here is the corresponding received header:
>
> X-Envelope-From: <us...@lanyon.com>
> Received: from S253906HZ1EW06.usstls6-hosting.savvis.net (unknown
> [209.16.192.170])
>
> Is it because there is no reverse DNS entry?
Yes.
Notice also that the rule checks the header From:, not the envelope,
and they could be different.
Joseph Brennan
Columbia University Information Technology
Re: Whitelist questions
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Wed, 2010-10-06 at 00:35 -0400, Alex wrote:
> > > We _really_ need to change that rule's description...
> >
> > Uhm, while I would never argue that naming to be unfortunate in
> > hindsight, despite most of the time actually matching its stated goal...
> >
> > I blame this one on Alex (the otherwise anonymous $mysqlstudent). He's
> > been around long enough, by far, to know about this. Just simply and
> > occasionally glimpsing threads on this list should have told him.
>
> Yes, my fault. I have experimented with it in the past on smaller
> systems, but never wanted to implement it on any system that was
> particularly critical based on what I've read here.
>
> I had recently implemented it on a smaller production system based on
> some documentation that I read outside of spamassassin.org, that also
> talked about using bayes with mysql.
>
> Hope that helps to explain what happened, and I'll be sure to read
> more thoroughly before implementing on my larger production systems.
Well, AWL is after all a rather dumb per-sender per-net-block score
averaging system. If it is suitable for you, depends.
Yes, it *can* average down an occasional spammy message, sent by an
otherwise known to be good sender. Its stated goal.
However, even if most spam does not re-use senders *and* net-blocks, it
still can average up (or down, the bad thing one notices eventually)
spam. Just as it can average up ham -- rarely noticed, unless it crosses
the threshold, though.
Again, a score averager. Likely to never have a bad influence on ham,
unless someone sent a GTUBE previously. Helpful, to counter the
occasional spammy message by a good sender.
Anything else is just the threshold crossing F[PN] you otherwise
wouldn't even have realized AWL fired on.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Whitelist questions
Posted by Alex <my...@gmail.com>.
Hi,
>> We _really_ need to change that rule's description...
>
> Uhm, while I would never argue that naming to be unfortunate in
> hindsight, despite most of the time actually matching its stated goal...
>
> I blame this one on Alex (the otherwise anonymous $mysqlstudent). He's
> been around long enough, by far, to know about this. Just simply and
> occasionally glimpsing threads on this list should have told him.
Yes, my fault. I have experimented with it in the past on smaller
systems, but never wanted to implement it on any system that was
particularly critical based on what I've read here.
I had recently implemented it on a smaller production system based on
some documentation that I read outside of spamassassin.org, that also
talked about using bayes with mysql.
Hope that helps to explain what happened, and I'll be sure to read
more thoroughly before implementing on my larger production systems.
Thanks,
Alex
Re: Whitelist questions
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Tue, 2010-10-05 at 13:09 -0700, John Hardin wrote:
> On Tue, 5 Oct 2010, Michael Scheidell wrote:
> > AWL is NOT an 'auto whitelist'. and is not used by default configs anymore.
> > instead of including the massive volume of documentation on what AWL is and
> > is not, just google.
>
> We _really_ need to change that rule's description...
Uhm, while I would never argue that naming to be unfortunate in
hindsight, despite most of the time actually matching its stated goal...
I blame this one on Alex (the otherwise anonymous $mysqlstudent). He's
been around long enough, by far, to know about this. Just simply and
occasionally glimpsing threads on this list should have told him.
I mean, for once, use this mailing list as a source of wisdom. Most of
us can learn from others' experience. Being subscribed to a mailing list
is not a post-only-if-question medium, but one to read. It's not a
fucking substitute for an easy google search, either.
Or our very own wiki. Ask it for "awl". Can you say AwlWrongWay!?
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Whitelist questions
Posted by John Hardin <jh...@impsec.org>.
On Tue, 5 Oct 2010, Michael Scheidell wrote:
> On 10/5/10 10:40 AM, Alex wrote:
>
>> * 0.7 AWL AWL: From: address is in the auto white-list
>>
>
> AWL is NOT an 'auto whitelist'. and is not used by default configs anymore.
> instead of including the massive volume of documentation on what AWL is and
> is not, just google.
We _really_ need to change that rule's description...
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
When designing software, any time you think to yourself "a user
would never be stupid enough to do *that*", you're wrong.
-----------------------------------------------------------------------
73 days until TRON Legacy
Re: Whitelist questions
Posted by Michael Scheidell <mi...@secnap.com>.
On 10/5/10 10:40 AM, Alex wrote:
> Hi,
>
> I have an email that I'm trying to whitelist using whitelist_from_rcvd
> and it's not working as I expect. I've created an entry:
>
> whitelist_from_rcvd user@lanyon.com savvis.net
>
> Here is the corresponding received header:
>
> X-Envelope-From:<us...@lanyon.com>
> Received: from S253906HZ1EW06.usstls6-hosting.savvis.net (unknown
> [209.16.192.170])
>
> Is it because there is no reverse DNS entry?
yes
> Also, am I somehow not using the AWL correctly? It's actually adding
> points and not subtracting them:
>
> * 0.0 RELAYCOUNTRY_US Relayed through United States
> * 0.3 LOC_RCVD_UNK Contains unknown IP in Received header
> * 1.4 BOTNET Relay might be a spambot or virusbot
> * [botnet0.8,ip=209.16.192.170,rdns=Lanyon.com,maildomain=lanyon.com,baddns]
> * 0.8 RDNS_NONE Delivered to internal network by a host with no rDNS
> * 1.0 FILL_THIS_FORM_LONG Fill in a form with personal information
> * 1.0 FILL_THIS_FORM Fill in a form with personal information
> * 1.5 FILL_THIS_FORM_FRAUD_PHISH Answer suspicious question(s)
> * 0.7 AWL AWL: From: address is in the auto white-list
>
> Under what circumstances would this happen?
AWL is NOT an 'auto whitelist'. and is not used by default configs
anymore.
instead of including the massive volume of documentation on what AWL is
and is not, just google.
--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
ISN: 1259*1300
> *| *SECNAP Network Security Corporation
* Certified SNORT Integrator
* 2008-9 Hot Company Award Winner, World Executive Alliance
* Five-Star Partner Program 2009, VARBusiness
* Best in Email Security,2010: Network Products Guide
* King of Spam Filters, SC Magazine 2008
______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________