Twist on Day Old Bread list idea

[Marc Perkel <ma...@perkel.com>]

I'm experimenting with a new list. Been testing it for a couple of 
months. Got a radical idea.

The problem with lists like Day Old Bread which lists new domains that 
spammers use is that there's a delay between when they are activated and 
when they are listed. It's just too hard to get a list of new domains. 
So - I'm trying the opposite approach. What I'm doing is listing 
existing domains and if it's not listed then it's new. So - here's how 
it works.

You query hostkarma.junkemailfilter.com

Not listed = new (new to us anyhow)
127.0.2.1 = last day
127.0.2.2 = last week
127.0.2.3 = older than a week

OK - so here's the rub. This catches 100% of all new domains. But - it 
will have false positives because if an old domain has never emailed 
anyone we filter for then it would also be considered new. We keep 40 
days of data. So - this list might be useful as long as it was combined 
with additional tests (probably spambot tests) as a score enhancer.

Let me know if you find this of interest.

Re: Twist on Day Old Bread list idea

[jp <jp...@saucer.midcoast.com>]

I think this would be a good DNS based list. It could have a slightly 
longer TTL than most DNS lists, as it's timeline would be generally 
pretty predictable. This would make the DNS caching an effective and 
efficient way to utilize the data.

I'd like to be able to implement it such as "if the nameservers of the 
domain aren't in my IP range, do this test".

On Wed, Dec 03, 2008 at 10:53:39AM -0500, Joseph Brennan wrote:
> 
> 
> --On Tuesday, December 2, 2008 12:23 -0800 Marc Perkel <ma...@perkel.com> 
> wrote:
> 
> 
> >You query hostkarma.junkemailfilter.com
> >
> >Not listed = new (new to us anyhow)
> >127.0.2.1 = last day
> >127.0.2.2 = last week
> >127.0.2.3 = older than a week
> >
> >OK - so here's the rub. This catches 100% of all new domains. But - it
> >will have false positives because if an old domain has never emailed
> >anyone we filter for then it would also be considered new. We keep 40
> >days of data. So - this list might be useful as long as it was combined
> >with additional tests (probably spambot tests) as a score enhancer.
> 
> 
> It's analogous to greylisting, to say that if we have not seen this
> domain in the past N days, we tempfail, or score, or something.
> 
> However I think it would be better to have a software package that
> implements this, rather than a remotely managed list, since each system
> would have its own set of domains that it sees frequently (or that it
> wants to whitelist permanently).
> 
> Joseph Brennan
> Columbia University Information Technology
> 
> 
> 

-- 
/*
Jason Philbrook   |   Midcoast Internet Solutions - Wireless and DSL
    KB1IOJ        |   Broadband Internet Access, Dialup, and Hosting 
 http://f64.nu/   |   for Midcoast Maine    http://www.midcoast.com/
*/

Re: Twist on Day Old Bread list idea

[Joseph Brennan <br...@columbia.edu>]

--On Tuesday, December 2, 2008 12:23 -0800 Marc Perkel <ma...@perkel.com> 
wrote:


> You query hostkarma.junkemailfilter.com
>
> Not listed = new (new to us anyhow)
> 127.0.2.1 = last day
> 127.0.2.2 = last week
> 127.0.2.3 = older than a week
>
> OK - so here's the rub. This catches 100% of all new domains. But - it
> will have false positives because if an old domain has never emailed
> anyone we filter for then it would also be considered new. We keep 40
> days of data. So - this list might be useful as long as it was combined
> with additional tests (probably spambot tests) as a score enhancer.


It's analogous to greylisting, to say that if we have not seen this
domain in the past N days, we tempfail, or score, or something.

However I think it would be better to have a software package that
implements this, rather than a remotely managed list, since each system
would have its own set of domains that it sees frequently (or that it
wants to whitelist permanently).

Joseph Brennan
Columbia University Information Technology