Automatic Whitelist Generation - Why wouldn't this work?

[Marc Perkel <ma...@perkel.com>]

OK - here's an idea I'm rolling around in my brain and thinking this 
could work to massively automatically generate white lists of IP 
addresses from companies that generate no spam at all. This could be 
used not only to greatly reduce false positives, but also you reduce 
system load. Any IP listed is ham and no need for further testing.

One thing that spammers can't spoof is RDNS. So if the RNDS of an IP is 
xxx.xxx.amd.com then we know the email is ham. Suppose that we start 
with a list of companies that we know that any email that comes from 
those hosts will always be ham then we can create a dynamically 
generated whitelist based on host IP addresses that come from the list.

A query comes in to a specially written DNS server where the RNDS is 
looked up and it's xxx.ibm.com and ibm.com is in the list of blessed ham 
hosts. We would need a fast way of getting rid of the subhost part to do 
the lookup, stripping the xxx part off to get the domain, . We would 
then return a yes response and cache the data in a local database.

The database could contain tens of thousands of domains that never send 
spam. How would we get this list? For now I'm doing it manually but it 
could possible be done by tracking ham and spam hist over time of 
verious IP addresses and looking for patterns of behavior that would 
indicate that indicate that the source is 100% clean.

Of course this wouldn't solve domains like yahoo, hotmail, comcast, and 
other mixed source spam but it would allow a lot of email to be 
preclassified as ham without further testing.

Who likes this idea?

Re: Automatic Whitelist Generation - Why wouldn't this work?

[Duane Hill <d....@yournetplus.com>]

On Mon, 25 Jun 2007, Marc Perkel wrote:

> Clarification. When I say that spammers can't spoof RNDS what I mean is that 
> if you do a reverse lookup and get a spoofed name then when you look up the 
> spoofed name it won't resolve back to the IP you looked up. I'm testing this 
> idea now.

RoadRunner Internet is already doing this. A customer of ours received a 
rejection message and this was within the content:

   452 Too many recipients received this hour.  Please see our rate limit policy at http://security.rr.com/spam.htm#ratelimit

I can't to it myself here. I had it set once and by the end of a day, I 
had received a number of complaints from customers that they were not 
receiving messages from who they were before.

Here I use Postfix and it is just a matter of "throwing a switch" 
so-to-speak to enable this feature.

Re: Automatic Whitelist Generation - Why wouldn't this work?

[Daniel J McDonald <da...@austinenergy.com>]

On Mon, 2007-06-25 at 06:25 -0700, Marc Perkel wrote:
> Clarification. When I say that spammers can't spoof RNDS what I mean is 
> that if you do a reverse lookup and get a spoofed name then when you 
> look up the spoofed name it won't resolve back to the IP you looked up. 
> I'm testing this idea now.

Of course, that's what the botnet plugin does.

But if you are looking for known ham sources, that's bonded sender or
some such.  They at least have a financial incentive to not send spam.
For anyone else it's just a matter of when they get pwn3d next.

-- 
Daniel J McDonald, CCIE # 2495, CISSP # 78281, CNX
Austin Energy
http://www.austinenergy.com

Re: Automatic Whitelist Generation - Why wouldn't this work?

[Marc Perkel <ma...@perkel.com>]

Clarification. When I say that spammers can't spoof RNDS what I mean is 
that if you do a reverse lookup and get a spoofed name then when you 
look up the spoofed name it won't resolve back to the IP you looked up. 
I'm testing this idea now.

Marc Perkel wrote:
> OK - here's an idea I'm rolling around in my brain and thinking this 
> could work to massively automatically generate white lists of IP 
> addresses from companies that generate no spam at all. This could be 
> used not only to greatly reduce false positives, but also you reduce 
> system load. Any IP listed is ham and no need for further testing.
>
> One thing that spammers can't spoof is RDNS. So if the RNDS of an IP 
> is xxx.xxx.amd.com then we know the email is ham. Suppose that we 
> start with a list of companies that we know that any email that comes 
> from those hosts will always be ham then we can create a dynamically 
> generated whitelist based on host IP addresses that come from the list.
>
> A query comes in to a specially written DNS server where the RNDS is 
> looked up and it's xxx.ibm.com and ibm.com is in the list of blessed 
> ham hosts. We would need a fast way of getting rid of the subhost part 
> to do the lookup, stripping the xxx part off to get the domain, . We 
> would then return a yes response and cache the data in a local database.
>
> The database could contain tens of thousands of domains that never 
> send spam. How would we get this list? For now I'm doing it manually 
> but it could possible be done by tracking ham and spam hist over time 
> of verious IP addresses and looking for patterns of behavior that 
> would indicate that indicate that the source is 100% clean.
>
> Of course this wouldn't solve domains like yahoo, hotmail, comcast, 
> and other mixed source spam but it would allow a lot of email to be 
> preclassified as ham without further testing.
>
> Who likes this idea?
>
>

Re: Automatic Whitelist Generation - Why wouldn't this work?

["John D. Hardin" <jh...@impsec.org>]

On Sun, 24 Jun 2007, Marc Perkel wrote:

> One thing that spammers can't spoof is RDNS. So if the RNDS of an
> IP is xxx.xxx.amd.com then we know the email is ham.

...unless, for instance, an AMD corporate box gets pwned.

> A query comes in to a specially written DNS server where the RNDS
> is looked up and it's xxx.ibm.com and ibm.com is in the list of
> blessed ham hosts. We would need a fast way of getting rid of the
> subhost part to do the lookup, stripping the xxx part off to get
> the domain, . We would then return a yes response and cache the
> data in a local database.

The owner of a netblock can put whatever they like in as the rDNS
hostname. They don't necessarily need to also own the domain they
claim it belongs to.

This means a spammer who owned a netblock could spoof whatever rDNS
they pleased; fortunately this is unlikely and would be really easy to
trap using a traditional DNSBL. On the flip side, DNS poisoning does 
exist, so a resourceful spammer may be able to poison rDNS to a 
degree.

> Of course this wouldn't solve domains like yahoo, hotmail,
> comcast, and other mixed source spam but it would allow a lot of
> email to be preclassified as ham without further testing.
> 
> Who likes this idea?

Basically you're suggesting a DNSWL.

Sounds like it has some merit. Nothing, however, is a panacaea.

--
 John Hardin KA7OHZ                    http://www.impsec.org/~jhardin/
 jhardin@impsec.org    FALaholic #11174     pgpk -a jhardin@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
  ...to announce there must be no criticism of the President or to
  stand by the President right or wrong is not only unpatriotic and
  servile, but is morally treasonous to the American public.
                                          -- Theodore Roosevelt, 1918
-----------------------------------------------------------------------
 10 days until The 231st anniversary of the Declaration of Independence