You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Marc Perkel <ma...@perkel.com> on 2009/10/24 00:33:33 UTC
Blacklists Compared 17 October 2009
http://www.sdsc.edu/~jeff/spam/cbc.html
The races!
Big upset for Baracudda as Spamhaus takes back the #1 position and Spam
Eating Monkey comes in second. (I don't count apews) Hostkarma pulls
ahead of Uceprotect who have been running neck and neck for 5th and 6th
place.
Re: Blacklists Compared 17 October 2009
Posted by Mariusz Kruk <Ma...@epsilon.eu.org>.
On Wednesday, 7 of April 2010, Marc Perkel wrote:
> Here's another good list that rates quality.
>
> http://www.intra2net.com/en/support/antispam/index.php
The methodology behind this rating is kinda peculiar.
What good is counting messages hit by lists? If I make a DNSBL which just
marks gmail as ham and everything else as spam, what accuracy would you
expect? Reasonably speaking, it should be inaccurate like hell but if you get
majority of mails from gmail, you'd get accuracy rate of 9x percent which
would be kinda silly. Repeating the same test is not giving any new
information.
--
/\-\/\-\/\-\/\-\/\-\/\-\/\
\ Kruk@epsilon.eu.org /
/ http://epsilon.eu.org/ \
\/-/\/-/\/-/\/-/\/-/\/-/\/
Re: Blacklists Compared 17 October 2009
Posted by Marc Perkel <su...@junkemailfilter.com>.
Here's another good list that rates quality.
http://www.intra2net.com/en/support/antispam/index.php
On 4/6/2010 7:41 PM, Alex wrote:
> Hi,
>
> Last October Marc posted the following URL that compared the various RBLs:
>
>
>> http://www.sdsc.edu/~jeff/spam/cbc.html
>>
> It seems barracuda is still leading, but is that also everyone's
> experience? Can anyone provide details on how Jeff computed this
> information and is it as cut-and-dried as this makes it seem? IOW,
> barracuda, the free service, is "better" than all the rest...
>
> Thanks,
> Alex
>
>
--
Marc Perkel - Sales/Support
support@junkemailfilter.com
http://www.junkemailfilter.com
Junk Email Filter dot com
415-992-3400
Re: Blacklists Compared 17 October 2009
Posted by Ralf Hildebrandt <Ra...@charite.de>.
* Ned Slider <ne...@unixmail.co.uk>:
> Last year when the barracuda config was first posted to this list, I
> implemented it on my personal mail server with a very high score so
> as to trigger automatic quarantines for all mail hitting the list,
> and have since checked all hits by hand. I currently use zen.spamhaus
> at the smtp stage to reject spam, so hits against barracuda only
> comprise of those that are "missed" by zen. I was particularly
> interested in FPs.
We use the same scheme
> During the last year I don't think I've seen a single FP hit against
> barracuda
I activated barracuda one week ago and already got 2 FPs. So one's MMV
--
Ralf Hildebrandt
Geschäftsbereich IT | Abteilung Netzwerk
Charité - Universitätsmedizin Berlin
Campus Benjamin Franklin
Hindenburgdamm 30 | D-12203 Berlin
Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
ralf.hildebrandt@charite.de | http://www.charite.de
Re: Blacklists Compared 17 October 2009
Posted by Mike Cardwell <sp...@lists.grepular.com>.
On 07/04/2010 12:01, corpus.defero wrote:
>> During the last year I don't think I've seen a single FP hit against
>> barracuda :surprised: That said, I still haven't found the confidence to
>> implement it at the smtp stage for outright rejection but the numbers
>> I'm seeing do tend towards telling me the list is of generally high quality.
>>
> In reality I make use of Barracuda first at SMTP time, Spamhaus after
> and have done so since 2008. I've never seen a FP from Barracuda in that
> time.
b.barracudacentral.org is amongst my top three lists these days. Along
with zen.spamhaus.org and bl.spamcop.net. There is no noticable
difference in the FP rate between them here and all three hit on a *lot*
of spam.
--
Mike Cardwell - Perl/Java/Web developer, Linux admin, Email admin
Read my tech Blog - https://secure.grepular.com/
Follow me on Twitter - http://twitter.com/mickeyc
Hire me - http://cardwellit.com/ http://uk.linkedin.com/in/mikecardwell
Re: Blacklists Compared 17 October 2009
Posted by "corpus.defero" <co...@idnet.com>.
On Wed, 2010-04-07 at 11:38 +0100, Ned Slider wrote:
> Alex wrote:
> > Hi,
> >
> > Last October Marc posted the following URL that compared the various RBLs:
> >
> >> http://www.sdsc.edu/~jeff/spam/cbc.html
> >
> > It seems barracuda is still leading, but is that also everyone's
> > experience? Can anyone provide details on how Jeff computed this
> > information and is it as cut-and-dried as this makes it seem? IOW,
> > barracuda, the free service, is "better" than all the rest...
> >
>
> As others have noted, FPs are not taken into account so one must
> consider that.
>
> Last year when the barracuda config was first posted to this list, I
> implemented it on my personal mail server with a very high score so as
> to trigger automatic quarantines for all mail hitting the list, and have
> since checked all hits by hand. I currently use zen.spamhaus at the smtp
> stage to reject spam, so hits against barracuda only comprise of those
> that are "missed" by zen. I was particularly interested in FPs.
>
> During the last year I don't think I've seen a single FP hit against
> barracuda :surprised: That said, I still haven't found the confidence to
> implement it at the smtp stage for outright rejection but the numbers
> I'm seeing do tend towards telling me the list is of generally high quality.
>
>
In reality I make use of Barracuda first at SMTP time, Spamhaus after
and have done so since 2008. I've never seen a FP from Barracuda in that
time.
I'm no fan of Barracuda - and that is widely documented. However, they
are a legal, professional business that is accountable. Spamhaus, on the
other hand, are not. Whilst their efforts in blocklisting are laudable
and noted, they appear to operate in a somewhat underground manner
without any proper base or contact details - not unlike gypsies.
Until they become fully legitimate and accountable their business
credibility will remain in question. With Barracuda, yes, you know they
are selling GPL code, you know that one of them is a former spammer, you
know about 'emailreg'. They make no secret of it. You don't, however,
know about just who is pulling the strings at Spamhaus. On several
occasions in the past I have received obvious and clear spam from the
likes of IHM in Nottingham, B2B deals, and uncounted attacks from
Emailvision in France all of which pass through Spamhaus and have you
saying 'Why is this?'
I have found (and I fully expect another round) that if you bad mouth or
question Spamhaus you are subjected to abuse, port scans, publication of
personal data in news groups and the like.
As far as the Barracuda list is concerned, I have total confidence in
it, and the company that operates it. Given the number of anti-spam
appliances they have in the field doing a very good job with it I would
say 'have confidence in it'.
Re: Blacklists Compared 17 October 2009
Posted by Ned Slider <ne...@unixmail.co.uk>.
Alex wrote:
> Hi,
>
> Last October Marc posted the following URL that compared the various RBLs:
>
>> http://www.sdsc.edu/~jeff/spam/cbc.html
>
> It seems barracuda is still leading, but is that also everyone's
> experience? Can anyone provide details on how Jeff computed this
> information and is it as cut-and-dried as this makes it seem? IOW,
> barracuda, the free service, is "better" than all the rest...
>
As others have noted, FPs are not taken into account so one must
consider that.
Last year when the barracuda config was first posted to this list, I
implemented it on my personal mail server with a very high score so as
to trigger automatic quarantines for all mail hitting the list, and have
since checked all hits by hand. I currently use zen.spamhaus at the smtp
stage to reject spam, so hits against barracuda only comprise of those
that are "missed" by zen. I was particularly interested in FPs.
During the last year I don't think I've seen a single FP hit against
barracuda :surprised: That said, I still haven't found the confidence to
implement it at the smtp stage for outright rejection but the numbers
I'm seeing do tend towards telling me the list is of generally high quality.
Re: Blacklists Compared 17 October 2009
Posted by "J.D. Falk" <jd...@cybernothing.org>.
On Apr 7, 2010, at 4:15 AM, Justin Mason wrote:
> he doesn't take FPs into account. this is a very serious problem with
> the methodology.
+1
--
J.D. Falk <jd...@returnpath.net>
Return Path Inc
Re: Blacklists Compared 17 October 2009
Posted by Justin Mason <jm...@jmason.org>.
he doesn't take FPs into account. this is a very serious problem with
the methodology.
--j.
On Wed, Apr 7, 2010 at 03:41, Alex <my...@gmail.com> wrote:
> Hi,
>
> Last October Marc posted the following URL that compared the various RBLs:
>
>> http://www.sdsc.edu/~jeff/spam/cbc.html
>
> It seems barracuda is still leading, but is that also everyone's
> experience? Can anyone provide details on how Jeff computed this
> information and is it as cut-and-dried as this makes it seem? IOW,
> barracuda, the free service, is "better" than all the rest...
>
> Thanks,
> Alex
>
>
Re: Blacklists Compared 17 October 2009
Posted by Kai Schaetzl <ma...@conactive.com>.
Alex wrote on Tue, 6 Apr 2010 22:41:18 -0400:
> > http://www.sdsc.edu/~jeff/spam/cbc.html
This list is not useable for choosing your preferred RBL. It doesn't take
false positives into account. apews.org for instance is not usable at all
and sorbs is only usable when you exclude their spamtraps list.
Kai
--
Get your web at Conactive Internet Services: http://www.conactive.com
Re: Blacklists Compared 17 October 2009
Posted by Marc Perkel <su...@junkemailfilter.com>.
On 4/7/2010 7:41 AM, corpus.defero wrote:
> On Wed, 2010-04-07 at 15:14 +0200, Raymond Dijkxhoorn wrote:
>
>> Hi!
>>
>>
>>>>> http://www.sdsc.edu/~jeff/spam/cbc.html
>>>>>
>>
>>>> It seems barracuda is still leading, but is that also everyone's
>>>> experience? Can anyone provide details on how Jeff computed this
>>>> information and is it as cut-and-dried as this makes it seem? IOW,
>>>> barracuda, the free service, is "better" than all the rest...
>>>>
>>
>>> spams him. So the experience of others might vary. There's not a lot of
>>> comparisons out there so this gives me some clue. But it doesn'y say anything
>>> about the quality of the lists as it has apews listed highly. If I created a
>>> list that blacklisted everything I would be first.
>>>
>> Setup a blacklist blocking ANY ip and you are ranked #1 in this test.
>> Its of no use at all IMHO.
>>
>> Bye,
>> Raymond.
>>
> They have - it's called 'UCEPROTECT' ......
>
>
>
Amen on the UCEPROTECT. What a bizarre company.
--
Marc Perkel - Sales/Support
support@junkemailfilter.com
http://www.junkemailfilter.com
Junk Email Filter dot com
415-992-3400
Re: Blacklists Compared 17 October 2009
Posted by "corpus.defero" <co...@idnet.com>.
On Wed, 2010-04-07 at 15:14 +0200, Raymond Dijkxhoorn wrote:
> Hi!
>
> >>> http://www.sdsc.edu/~jeff/spam/cbc.html
>
> >> It seems barracuda is still leading, but is that also everyone's
> >> experience? Can anyone provide details on how Jeff computed this
> >> information and is it as cut-and-dried as this makes it seem? IOW,
> >> barracuda, the free service, is "better" than all the rest...
>
> > spams him. So the experience of others might vary. There's not a lot of
> > comparisons out there so this gives me some clue. But it doesn'y say anything
> > about the quality of the lists as it has apews listed highly. If I created a
> > list that blacklisted everything I would be first.
>
> Setup a blacklist blocking ANY ip and you are ranked #1 in this test.
> Its of no use at all IMHO.
>
> Bye,
> Raymond.
They have - it's called 'UCEPROTECT' ......
Re: Blacklists Compared 17 October 2009
Posted by Alex <my...@gmail.com>.
Hi,
> Independent testing like the VB tests tell me much more.
>
> http://www.virusbtn.com/virusbulletin/archive/2010/03/vb201003-vbspam-comparative
>
> And yes that more or less the commercial products, but it shows also how
> lits like SURBL perform. But also ratings of the large vendors. And the FP
> levels.
Yes, that is good information. Sure appreciate your feedback.
> There is much more to do when rating things then just do some simple
> counting. The scoring in SA is done pretty well (compliments!) but even
> there are gaps and holes like pointed out.
I'm hoping to stay on topic with the RBLs and as they pertain to SA,
so I would also like to determine how to make an educated decision.
I've done some further research, and have found a few more useful
comparisons to add to the list:
http://spamlinks.net/filter-dnsbl-lists.htm
At the bottom of this page is a section that lists other comparisons.
It would be nice to know how current they are, however. The link also
contains a handful of neat command-line tools and libraries that can
be used to query the RBLs.
I'm especially interested in using multi.uribl.com and
zen.spamhaus.org at SMTP restriction time. However, it seems like zen
is the one that is most popular. Should uribl.com not be used for
this?
At SMTP time, are there others that people should consider using?
This would be really helpful, because there doesn't seem to be any
authoritative info out there about achieving the best configuration,
outside of simple trial-and-error.
Thanks,
Alex
Re: Blacklists Compared 17 October 2009
Posted by Raymond Dijkxhoorn <ra...@prolocation.net>.
Hi!
>> Setup a blacklist blocking ANY ip and you are ranked #1 in this test.
>> Its of no use at all IMHO.
> Yes, certainly, and I guess it was a loaded question of me to ask,
> because it was almost too obvious that I thought I was missing
> something. I don't think it's _completely_ useless though, because the
> two leaders are for the most part reputable and don't just block ANY
> IP.
> Raymond, I remember your name from reading at surbl.org (actually, you
> must go to www.surbl.org, heh). Is this the list you recommend?
I dont recommend any list. Its up to anyone to pick and mix. Some list
work for the other, some dont. My personal experience with the cuda list
is not good. That doesnt mean it will work for others. That wasnt the
story at all. You asked if this comparison is a benchmark, it is, but
doesnt say anything about FP rating or effectiveness so its just a
number. A rather useless number also IMHO.
> I somewhat agree with the comment about barracuda being accountable,
> but mxtools is apparently running the business side of things at
> SURBL, and for large systems you need to pay, so I would assume they
> want you to come back to renew.
Not interested to talk about that. You asked about comparison, not about
pricing or surbl or mxtools or whatever. Please check my e-mail from thats
used on the list. I post on personal title here. If you wanna post or talk
to mxtools please try another list ;) Its offtopic.
> I'm sure searching could probably eventually find something, but does
> anyone know or have links that show the surbl and barracuda motivation
> behind their service? Of course it is to block spam, but what does
> barracuda expect to gain by making it free, for example, where surbl
Simple, to stop spam? For SURBL, the main reason was to help people out.
And we do that and have done that for allmost 8 years now.
We moved after giving away free service, and years of free time. Yes a
blacklist doesnt compile just like that... That didnt turn out for the
long term so we switched models. All is explained on different lists and
if you like to know, check there. We basicly wanted to secure service
level and existance for over time.
There is a lot of talks about Cuda, ask the people from Spamhaus or
any some of the open source vendor who's code is used on the Cuda devices,
on the SA list are enough people who can help you out with getting some
clue whats going on. I wont poke around in that fire, you get burned
easilly. So this is all you get ;)
> An occasional FP is almost equally as subjective as anything else, and
> a difficult metric in which to place any comparison value.
>
> Would a more detailed and thorough comparison be any use? IOW, number
> of changes per day, how many total entries, etc, or is everyone okay
> with how it is currently?
Independent testing like the VB tests tell me much more.
http://www.virusbtn.com/virusbulletin/archive/2010/03/vb201003-vbspam-comparative
And yes that more or less the commercial products, but it shows also how
lits like SURBL perform. But also ratings of the large vendors. And the
FP levels.
Even the SA corpus, where we have a open ticket about the ratings isnt
currently ok with rating. Score is calculated over time but domains
expire. Its hard... retesting doesnt give a full solution for that either.
See:
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6386
There is much more to do when rating things then just do some simple
counting. The scoring in SA is done pretty well (compliments!) but even
there are gaps and holes like pointed out.
So its certainly not something you can put aside in just a 5 mins talk.
bye,
Raymond.
Re: Blacklists Compared 17 October 2009
Posted by Alex <my...@gmail.com>.
Hi,
>>>> http://www.sdsc.edu/~jeff/spam/cbc.html
[...]
> Setup a blacklist blocking ANY ip and you are ranked #1 in this test.
> Its of no use at all IMHO.
Yes, certainly, and I guess it was a loaded question of me to ask,
because it was almost too obvious that I thought I was missing
something. I don't think it's _completely_ useless though, because the
two leaders are for the most part reputable and don't just block ANY
IP.
Raymond, I remember your name from reading at surbl.org (actually, you
must go to www.surbl.org, heh). Is this the list you recommend?
I somewhat agree with the comment about barracuda being accountable,
but mxtools is apparently running the business side of things at
SURBL, and for large systems you need to pay, so I would assume they
want you to come back to renew.
I'm sure searching could probably eventually find something, but does
anyone know or have links that show the surbl and barracuda motivation
behind their service? Of course it is to block spam, but what does
barracuda expect to gain by making it free, for example, where surbl
charges for theirs? They simply hope people will contribute back to it
to make it better for their appliance customers?
An occasional FP is almost equally as subjective as anything else, and
a difficult metric in which to place any comparison value.
Would a more detailed and thorough comparison be any use? IOW, number
of changes per day, how many total entries, etc, or is everyone okay
with how it is currently?
Thanks,
Alex
Re: Blacklists Compared 17 October 2009
Posted by Raymond Dijkxhoorn <ra...@prolocation.net>.
Hi!
>>> http://www.sdsc.edu/~jeff/spam/cbc.html
>> It seems barracuda is still leading, but is that also everyone's
>> experience? Can anyone provide details on how Jeff computed this
>> information and is it as cut-and-dried as this makes it seem? IOW,
>> barracuda, the free service, is "better" than all the rest...
> spams him. So the experience of others might vary. There's not a lot of
> comparisons out there so this gives me some clue. But it doesn'y say anything
> about the quality of the lists as it has apews listed highly. If I created a
> list that blacklisted everything I would be first.
Setup a blacklist blocking ANY ip and you are ranked #1 in this test.
Its of no use at all IMHO.
Bye,
Raymond.
Re: Blacklists Compared 17 October 2009
Posted by Marc Perkel <su...@junkemailfilter.com>.
On 4/6/2010 7:41 PM, Alex wrote:
> Hi,
>
> Last October Marc posted the following URL that compared the various RBLs:
>
>
>> http://www.sdsc.edu/~jeff/spam/cbc.html
>>
> It seems barracuda is still leading, but is that also everyone's
> experience? Can anyone provide details on how Jeff computed this
> information and is it as cut-and-dried as this makes it seem? IOW,
> barracuda, the free service, is "better" than all the rest...
>
> Thanks,
> Alex
>
>
I don't know the details but from what I understand it is a raw count of
who spams him. So the experience of others might vary. There's not a lot
of comparisons out there so this gives me some clue. But it doesn'y say
anything about the quality of the lists as it has apews listed highly.
If I created a list that blacklisted everything I would be first.
--
Marc Perkel - Sales/Support
support@junkemailfilter.com
http://www.junkemailfilter.com
Junk Email Filter dot com
415-992-3400
Re: Blacklists Compared 17 October 2009
Posted by Alex <my...@gmail.com>.
Hi,
Last October Marc posted the following URL that compared the various RBLs:
> http://www.sdsc.edu/~jeff/spam/cbc.html
It seems barracuda is still leading, but is that also everyone's
experience? Can anyone provide details on how Jeff computed this
information and is it as cut-and-dried as this makes it seem? IOW,
barracuda, the free service, is "better" than all the rest...
Thanks,
Alex