You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Marc Perkel <ma...@perkel.com> on 2006/06/30 23:26:27 UTC

White List and Yellow List DNS Servers - Proposal

I want to propose and idea that I've been testing with some success. But 
there are plenty of people who are a lot sharper than I am that can 
implement it better. Here's what I'm thinking.

We are all familiar with DNS blacklists to block spam. But what about 
lists of other servers? What about white lists servers that never send 
spam. Or - a new term, Yellow list, servers that send mostly nonspam but 
send some spam. These ar servers that should never be blacklisted.

Spammers can fake a lot of things but that can't fake the host they are 
sending from. So - the way I see it, there are 3 kinds of hosts.

1) Blacklist - host that send only spam
2) Whitelist - hosts that never send spam.
3) Yellowlist - Hosts that have some spam sneak through but should never 
be blacklisted. gmail, yahoo, hotmail, etc are examples.

The problem with blacklists is false positives. If not for that we could 
use them more to block spam. So - with white and yellow lists we can 
check then we can make sure that good servers are never blacklisted. 
That makes the blacklisting more accurate. The processing would go as 
follows:

1) Whitelisted servers can bypass spam filtering.
2) Yellow listed server would bypass blacklist testing.
3) Blacklisted servers are bounced without spam filtering
4) Yellow and Unlisted email is processed with Spam Assassin for spam 
testing
5) Statistical feedback on hosts to the various lists.

I've implemented a crude version of this model and it's working very 
well. By tracking ham I can make sure that hosts that send ham are never 
blacklisted. This has significantly increased the accuracy of my 
blacklists virtually eliminating false positives. So - I'm looking to 
bring together some of the great minds here and let's build a system 
right that can be used to block spam and greatly reduce the server loads 
on SA by bypassing SA processing.

Who likes this idea?


Re: White List and Yellow List DNS Servers - Proposal

Posted by Kevin Golding <ke...@caomhin.demon.co.uk>.
In article <44...@perkel.com>, Marc Perkel <ma...@perkel.com>
writes
>The idea of this is to improve the blacklisting be creating lists to 
>reduce false positives. For example, spamcop has a problem were they 
>sometimes block earthlink and google servers because those servers send 
>some spam. Using these yellow lists can prevent spamcop from listing 
>servers that are known to be servers that should never be blacklisted.

I was under the impression that Spamcop didn't regard listing
earthlink/gmail/hotmail/foo as a problem.  They're smart enough to have
their own internal whitelists that they can handle as they see fit -
those servers don't exactly get listed at the first sign of trouble so
it seems somewhat deliberate to me.

You may find a personal whitelist more suited to your needs.  Either use
it as a guide to not bother doing RBL checks for a message but still
process other rules or use it as a meta to 0 out any RBL hit you find.

Kevin

Re: White List and Yellow List DNS Servers - Proposal

Posted by Marc Perkel <ma...@perkel.com>.

Bart Schaefer wrote:
> On 6/30/06, Marc Perkel <ma...@perkel.com> wrote:
>>
>> Yeah - but what I'm thinking of is something that is automatic and
>> reputation based rather that paying someone to certify you. In other
>> words your server get whitelisted because you never send spam.
>
> Paid or otherwise, how do you get on the list in the first place?  You
> obviously used some criteria based on your own server logs to
> determine which IPs "never" send spam -- but "never" is a long time,
> and in some cases "spam" is objective (people report all kinds of
> stuff as spam for all kinds of reasons).

One of the things I'm doing is tracking spam/ham by host IP address. I 
count that ham/spam ratio and the IP addresses that get 99%+ ham (many 
are 100%) get whitelisted. Those that are 99%+ spam are blacklisted. 
Those that send mostly ham but some spam are yellow listed which 
prevents them from being blacklisted.

So the idea would be to have some trusted filtering services that are 
blessed to be able to report the IP address of spam and ham to a central 
DNS list that would keep count. Those hosts with a (near) perfect 
reputation are whitelisted. Those with a good reputation are 
yellowlisted. And of course we continue with blacklisting as we do now.

The idea of this is to improve the blacklisting be creating lists to 
reduce false positives. For example, spamcop has a problem were they 
sometimes block earthlink and google servers because those servers send 
some spam. Using these yellow lists can prevent spamcop from listing 
servers that are known to be servers that should never be blacklisted.



Re: White List and Yellow List DNS Servers - Proposal

Posted by Bart Schaefer <ba...@gmail.com>.
On 6/30/06, Marc Perkel <ma...@perkel.com> wrote:
>
> Yeah - but what I'm thinking of is something that is automatic and
> reputation based rather that paying someone to certify you. In other
> words your server get whitelisted because you never send spam.

Paid or otherwise, how do you get on the list in the first place?  You
obviously used some criteria based on your own server logs to
determine which IPs "never" send spam -- but "never" is a long time,
and in some cases "spam" is objective (people report all kinds of
stuff as spam for all kinds of reasons).

Re: White List and Yellow List DNS Servers - Proposal

Posted by Marc Perkel <ma...@perkel.com>.

Bart Schaefer wrote:
> On 6/30/06, Marc Perkel <ma...@perkel.com> wrote:
>> Who likes this idea?
>
> Evidently habeas.com does, as that's now their business model.  Also
> Bonded Sender (I think they changed the name recently, but I forget to
> what).  And I believe the ISIPP maintains several such lists.  Do a
> Google on "reputation service".
>

Yeah - but what I'm thinking of is something that is automatic and 
reputation based rather that paying someone to certify you. In other 
words your server get whitelisted because you never send spam.

Re: White List and Yellow List DNS Servers - Proposal

Posted by Bart Schaefer <ba...@gmail.com>.
On 6/30/06, Marc Perkel <ma...@perkel.com> wrote:
> Who likes this idea?

Evidently habeas.com does, as that's now their business model.  Also
Bonded Sender (I think they changed the name recently, but I forget to
what).  And I believe the ISIPP maintains several such lists.  Do a
Google on "reputation service".