Re: [SA] DNSBL Comparison 20091010

[Adam Katz <an...@khopis.com>]

Henrik K wrote:
> On Sun, Oct 11, 2009 at 01:19:47AM -0400, Adam Katz wrote:
>> *Especially* while DNSWLs lack an abuse-reporting mechanism.
>>
>> I have seen SO much DNSWL'd spam that I've had to migrate to using
> 
> Just to be clear, what DNSWLs are you talking about? It's a bit
> confusing as the official DNSWL is called "DNSWL". While it
> doesn't(?) have an automated "abuse-reporting mechanism", it sure
> accepts such reports.
> 
> Maybe it's just me, but there is currently only one proven DNSWL.

Here are the default scores for the DNSWLs I know of:

RCVD_IN_DNSWL_LOW 0 -1 0 -1
RCVD_IN_DNSWL_MED 0 -4 0 -4
RCVD_IN_DNSWL_HI 0 -8 0 -8
RCVD_IN_HOSTKARMA_W -5 # (nondefault rule, Marc's suggested score)
RCVD_IN_BSP_TRUSTED 0 -4.3 0 -4.3
RCVD_IN_IADB_DOPTIN_GT50 0
RCVD_IN_IADB_ML_DOPTIN 0 -6 0 -6
RCVD_IN_IADB_VOUCHED 0 -2.2 0 -2.2
RCVD_IN_SSC_TRUSTED_COI 0 -3.7 0 -3.7

I've had myriads of falsely whitelisted messages hit DNSWL (.org) and
HOSTKARMA_W (JMF_W).  SpamCop's reporting mechanism has reporting such
spam to BSP built in.  I'd /love/ to see DNSWL and Marc hook in there
too (and/or gain direct access to SA's own reporting mechanism).

Re: [SA] DNSBL Comparison 20091010

[Adam Katz <an...@khopis.com>]

Matthias Leisi wrote (accidentally off-list):
> Adam Katz schrieb:
> 
>> My last report was sent at 2009-04-10 17:50:30 UTC to admin@dnswl.org
>> with subject "Suggested Change DNSWL Id 3523"
> 
> That's cvent-planner.com. Based on your report and others we received,
> we lowered the score for their entries to "none" on 2009-06-24.
> 
>> Okay, I'll resume reporting.  abuse@dnswl.org or admin@dnswl.org (or
>> something else)?
> 
> admins /at/ dnswl.org

Ah, I'll note that.  Now that I know that you're actively accepting such
reports, I'll be more attentive about submitting them.  Thanks!

>>> I'd love that too, but SpamCop has denied my multiple requests over the
>>> years.
>> Dang.  Given how that should be a trivial implementation, that's a
>> surprise.  I seem to recall their addition of BSP being somewhat recent
>> ... maybe you haven't asked since before they added support for that
>> sort of thing?
> 
> I'll give it another go. If anyone has a good working contact at
> Spamcop, I'd appreciate some support :)

I think the problem is that they're short-staffed.  I had a good dialog
in April with Don D'Minion at service/at/admin.spamcop.net, but I'm not
sure if he's at the right level for this request (and that's not exactly
a personal address).

Re: [SA] DNSBL Comparison 20091010

[Adam Katz <an...@khopis.com>]

Matthias Leisi wrote:
> Did you report them to us? If there are *myriads*, there must be some
> serious error which we need to fix (IPs/ranges falsely listed,
> inappropriate trust levels listed, sometimes also errors in eg how
> trusted_networks are set up).

My last report was sent at 2009-04-10 17:50:30 UTC to admin@dnswl.org
with subject "Suggested Change DNSWL Id 3523"

Okay, I'll resume reporting.  abuse@dnswl.org or admin@dnswl.org (or
something else)?

>> HOSTKARMA_W (JMF_W).  SpamCop's reporting mechanism has reporting such
>> spam to BSP built in.  I'd /love/ to see DNSWL and Marc hook in there
> 
> I'd love that too, but SpamCop has denied my multiple requests over the
> years.

Dang.  Given how that should be a trivial implementation, that's a
surprise.  I seem to recall their addition of BSP being somewhat recent
... maybe you haven't asked since before they added support for that
sort of thing?

Re: [SA] DNSBL Comparison 20091010

[Matthias Leisi <ma...@leisi.net>]

Adam Katz schrieb:

> I've had myriads of falsely whitelisted messages hit DNSWL (.org) and

Did you report them to us? If there are *myriads*, there must be some
serious error which we need to fix (IPs/ranges falsely listed,
inappropriate trust levels listed, sometimes also errors in eg how
trusted_networks are set up).

> HOSTKARMA_W (JMF_W).  SpamCop's reporting mechanism has reporting such
> spam to BSP built in.  I'd /love/ to see DNSWL and Marc hook in there

I'd love that too, but SpamCop has denied my multiple requests over the
years.

-- Matthias, speaking for dnswl.org

Re: White lists and white rules

[Ted Mittelstaedt <te...@ipinc.net>]

Marc Perkel wrote:
> 
> 
> Henrik K wrote:
>> On Sun, Oct 11, 2009 at 01:10:17PM -0400, Adam Katz wrote:
>>   
>>> Here are the default scores for the DNSWLs I know of:
>>>
>>> RCVD_IN_DNSWL_LOW 0 -1 0 -1
>>> RCVD_IN_DNSWL_MED 0 -4 0 -4
>>> RCVD_IN_DNSWL_HI 0 -8 0 -8
>>> RCVD_IN_HOSTKARMA_W -5 # (nondefault rule, Marc's suggested score)
>>>     
>>
>> You have to remember that Marc is automated and as such can never be as
>> fully trusted as DNSWL which has a completely different listing criteria. I
>> have to admit W is getting better though, nearing DNSWL_LOW rates. I'd like
>> to see many levels of W though and not just a single lump.
>>
>>   
>>> RCVD_IN_BSP_TRUSTED 0 -4.3 0 -4.3
>>> RCVD_IN_IADB_DOPTIN_GT50 0
>>> RCVD_IN_IADB_ML_DOPTIN 0 -6 0 -6
>>> RCVD_IN_IADB_VOUCHED 0 -2.2 0 -2.2
>>> RCVD_IN_SSC_TRUSTED_COI 0 -3.7 0 -3.7
>>>     
>>
>> These practically don't exist, if you look at ruleqa.
>>
>>   
> For what it's worth there are really only 3 serious white lists on the 
> planet. I'm surprised no one is
> testing the emailreg list. 

I'm not.

> There are dozens of black lists. Doing white 
> lists is actually easier than doing
> black lists because there are thousands of servers out there that send 
> nothing but good email. That have
> good FcRDNS, they are static, and unlike the black lists IPs they aren't 
> trying to be evasive. It's low
> hanging fruit. On my servers if you are white listed your message just 
> sails through the system.
> 
> One of my beefs with the spam filtering community is that there is too 
> much focus on detecting spam and not
> enough focus on detecting non-spam. We need more white lists and we need 
> more white rules.

All of that is quite true.

> A lot of what
> I'm doing is because no one else is doing it. I'd love it if other 
> people would get into the white list
> business and do a better job than me. I'm really good at coming up with 
> new and original ideas but others
> are usually better at implementing them.
> 
> I'd love to have sources of IPs that send nothing but good email. It 
> would be trivial to set up a system to
> detect that and to collate results for several trusted reporters. If I 
> have some people who were interested
> we could set something up and a lot of good email could sail through the 
> system with better accuracy.
> 

Human nature is to expend effort going after the people who cause 
problems rather than giving "Attaboys" to the people who aren't.

Be patient, though.  We still have not got to the point that adding
spam detection rules to SA is in the region of diminishing returns.
Spammers by and large are still dumb as posts and sending mail that
a child of 10 can recognize as spam.

Give it another decade, when the day comes that you can no longer
detect most spam merely by reading the subject line, but instead you
have to open it, that's when more attention will be paid to this issue.

Ted

Re: White lists and white rules

["J.D. Falk" <jd...@cybernothing.org>]

Ted Mittelstaedt wrote:

> Thus, any reputable blacklist service will ALSO need to constantly
> monitor to make sure that any IP that's listed still deserves to be
> there.

Absolutely.  I keep forgetting that anyone would think otherwise; major spam 
sources haven't been stationary in years.

-- 
J.D. Falk
Return Path Inc
http://www.returnpath.net/

Re: White lists and white rules

[Ted Mittelstaedt <te...@ipinc.net>]

J.D. Falk wrote:
> Aaron Wolfe wrote:
> 
>>> Not true. There are servers that say send out bank statements and 
>>> 100% of
>>> what it sends is bank statements.
>>
>> Until the day those servers get hacked, or they take on a new client
>> who sends a different type of mail, etc.
> 
> That's why any serious 3rd party whitelist service will constantly 
> monitor to make sure that any IP that's listed still deserves to be there.
> 

The same issues apply equally to both kinds of services.

One of the big concerns today is we are running out of IP addresses
and most networks are not ready to switch to IP version 6 yet.  Runout
of "virgin" never-assigned IPv4 addresses will occur within 2 years
and for a number of years following there will be a large effort
made to recover abandoned IPv4 addresses.  Many of those addresses
are occupied today by squatters who have gotten disreputable ISP's
to announce them, and are using them to spam.  Within 2-3 years those 
addresses will be reassigned to ISPs who will be cleaning them up,
and will be wanting to use them legitimately.

Thus, any reputable blacklist service will ALSO need to constantly
monitor to make sure that any IP that's listed still deserves to be
there.


Ted

Re: White lists and white rules

["J.D. Falk" <jd...@cybernothing.org>]

Aaron Wolfe wrote:

>> Not true. There are servers that say send out bank statements and 100% of
>> what it sends is bank statements.
> 
> Until the day those servers get hacked, or they take on a new client
> who sends a different type of mail, etc.

That's why any serious 3rd party whitelist service will constantly monitor 
to make sure that any IP that's listed still deserves to be there.

-- 
J.D. Falk
Return Path Inc
http://www.returnpath.net/

Re: White lists and white rules

[Aaron Wolfe <aa...@gmail.com>]

On Mon, Oct 12, 2009 at 11:50 AM, Marc Perkel <ma...@perkel.com> wrote:
>
>
> Warren Togami wrote:
>>
>> On 10/12/2009 09:18 AM, Marc Perkel wrote:
>>>
>>> For what it's worth there are really only 3 serious white lists on the
>>> planet. I'm surprised no one is
>>> testing the emailreg list. There are dozens of black lists. Doing white
>>> lists is actually easier than doing
>>> black lists because there are thousands of servers out there that send
>>> nothing but good email. That have
>>> good FcRDNS, they are static, and unlike the black lists IPs they aren't
>>> trying to be evasive. It's low
>>> hanging fruit. On my servers if you are white listed your message just
>>> sails through the system.
>>
>> This seems to me like a naive system.  Even the best networks that send
>> nothing but ham will occasionally have an infected spambot.
>>
>> BTW, how do I report HOSTKARMA W failures?
>>
>> Warren
>>
>
> Not true. There are servers that say send out bank statements and 100% of
> what it sends is bank statements.
>

Until the day those servers get hacked, or they take on a new client
who sends a different type of mail, etc.

Re: White lists and white rules

[Marc Perkel <ma...@perkel.com>]

Warren Togami wrote:
> On 10/12/2009 09:18 AM, Marc Perkel wrote:
>> For what it's worth there are really only 3 serious white lists on the
>> planet. I'm surprised no one is
>> testing the emailreg list. There are dozens of black lists. Doing white
>> lists is actually easier than doing
>> black lists because there are thousands of servers out there that send
>> nothing but good email. That have
>> good FcRDNS, they are static, and unlike the black lists IPs they aren't
>> trying to be evasive. It's low
>> hanging fruit. On my servers if you are white listed your message just
>> sails through the system.
>
> This seems to me like a naive system.  Even the best networks that 
> send nothing but ham will occasionally have an infected spambot.
>
> BTW, how do I report HOSTKARMA W failures?
>
> Warren
>

Not true. There are servers that say send out bank statements and 100% 
of what it sends is bank statements.

As to reporting failures, email me the list.

Re: White lists and white rules

[Warren Togami <wt...@redhat.com>]

On 10/12/2009 09:18 AM, Marc Perkel wrote:
> For what it's worth there are really only 3 serious white lists on the
> planet. I'm surprised no one is
> testing the emailreg list. There are dozens of black lists. Doing white
> lists is actually easier than doing
> black lists because there are thousands of servers out there that send
> nothing but good email. That have
> good FcRDNS, they are static, and unlike the black lists IPs they aren't
> trying to be evasive. It's low
> hanging fruit. On my servers if you are white listed your message just
> sails through the system.

This seems to me like a naive system.  Even the best networks that send 
nothing but ham will occasionally have an infected spambot.

BTW, how do I report HOSTKARMA W failures?

Warren

Re: [SA] DNSBL Comparison 20091010

[Henrik K <he...@hege.li>]

On Sun, Oct 11, 2009 at 01:10:17PM -0400, Adam Katz wrote:
> 
> Here are the default scores for the DNSWLs I know of:
> 
> RCVD_IN_DNSWL_LOW 0 -1 0 -1
> RCVD_IN_DNSWL_MED 0 -4 0 -4
> RCVD_IN_DNSWL_HI 0 -8 0 -8
> RCVD_IN_HOSTKARMA_W -5 # (nondefault rule, Marc's suggested score)

You have to remember that Marc is automated and as such can never be as
fully trusted as DNSWL which has a completely different listing criteria. I
have to admit W is getting better though, nearing DNSWL_LOW rates. I'd like
to see many levels of W though and not just a single lump.

> RCVD_IN_BSP_TRUSTED 0 -4.3 0 -4.3
> RCVD_IN_IADB_DOPTIN_GT50 0
> RCVD_IN_IADB_ML_DOPTIN 0 -6 0 -6
> RCVD_IN_IADB_VOUCHED 0 -2.2 0 -2.2
> RCVD_IN_SSC_TRUSTED_COI 0 -3.7 0 -3.7

These practically don't exist, if you look at ruleqa.