You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Per Jessen <pe...@computer.org> on 2007/12/17 18:24:38 UTC
blackholes.us ?
Does anyone have a current status for blackholes.us ? The rsync'ed data
is about 18months old.
I had an email rejected earlier today due to a server
being "blacklisted" by germany.blackholes.us ....
/Per Jessen, Zürich
Re: blackholes.us ?
Posted by Per Jessen <pe...@computer.org>.
Matt Kettler wrote:
> Per Jessen wrote:
>>> True. Although this is the SpamAssassin mailing list, so, within
>>> context of scored systems, the blackholes.us might be of some use.
>>> That said, if you're using SA, it's much lighter weight to use the
>>> RelayCountries plugin.
>>
>> That seems hard to believe - lighter than a single local DNS query?
>>
> Single local query of a database. Not everyone using blackholes.us is
> doing zone transfers, so for some folks it's heavy-weight because it's
> an off-site DNS query.
Yeah, unless you serve the blackholes list locally, it's slow. (you
don't need to do zone-transfers).
> When compared to someone using zone transfers, it's still likely to be
> marginally faster because the Ip::Country::Fast code and database are
> both optimized for doing this kind of query very quickly.
I don't know the RelayCountries plugin nor the code, but I have
difficulty believing that querying any list served by rbldnsd would be
any slower.
/Per Jessen, Zürich
Re: Ip::Country::Fast performance (was: blackholes.us?)
Posted by Per Jessen <pe...@computer.org>.
Matt Kettler wrote:
> When compared to someone using zone transfers, it's still likely to be
> marginally faster because the Ip::Country::Fast code and database are
> both optimized for doing this kind of query very quickly.
You're right - I ran some tests, and Ip::Country::Fast is not just
marginally faster, it's up to 10 times faster than doing local DNS
lookups:
On a relatively low-end machine (Athlon 1.2GHz, 768M RAM):
Ip::Country::Fast - 66000 lookups took 8sec, 839000 lookups took 73sec.
local DNS lookup (php) - 66000 lookups = 85sec, 839000 lookups took
25min
/Per Jessen, Zürich
Re: blackholes.us ?
Posted by mouss <mo...@netoyen.net>.
Per Jessen wrote:
> Matt Kettler wrote:
>
>
>>> That's easily checked - we could run a comparison of any up-to-date
>>> geoip database against blackholes.us.
>>>
>>>
>> True.
>>
>
> Well, I've answered my own question. I ran a test of maxmind addresses
> dated 2007/04/04 against the blackholes.us data dated 2006/05/24. The
> maxmind database had 66231 entries/ranges. I took the first of each
> one and tested it on blackholes.us.
>
> Total runtime was 2m5s = 1.9ms per lookup.
>
> It returned 16300 maxmind entries also found in blackholes.us, but also
> 49931 not found. I think the staleness of blackholes.us has been
> confirmed.
>
No. nothing confirmed by this "test". you need to find an IP in
blackholes.us for which the country is wrong.
anyway, you can try nerd.dk...
Re: blackholes.us ?
Posted by Matt Kettler <mk...@verizon.net>.
Per Jessen wrote:
> Matt Kettler wrote:
>
>
>>> That's easily checked - we could run a comparison of any up-to-date
>>> geoip database against blackholes.us.
>>>
>>>
>> True.
>>
>
> Well, I've answered my own question. I ran a test of maxmind addresses
> dated 2007/04/04 against the blackholes.us data dated 2006/05/24. The
> maxmind database had 66231 entries/ranges. I took the first of each
> one and tested it on blackholes.us.
>
> Total runtime was 2m5s = 1.9ms per lookup.
>
> It returned 16300 maxmind entries also found in blackholes.us, but also
> 49931 not found. I think the staleness of blackholes.us has been
> confirmed.
>
Again, I'm less interested in staleness than in accuracy, particularly
"false-positive" type errors. ie: mis-representing an IP address as
belonging in some particular region when it does not.
Really what you've proven here is lack of completeness, which might or
might not be due to staleness. (I strongly suspect maxminds database was
probably larger and more complete even if you used the 2006/05/24
version of maxmind)..
>
> /Per Jessen, Zürich
>
>
>
Re: blackholes.us ?
Posted by Per Jessen <pe...@computer.org>.
Matt Kettler wrote:
>> That's easily checked - we could run a comparison of any up-to-date
>> geoip database against blackholes.us.
>>
> True.
Well, I've answered my own question. I ran a test of maxmind addresses
dated 2007/04/04 against the blackholes.us data dated 2006/05/24. The
maxmind database had 66231 entries/ranges. I took the first of each
one and tested it on blackholes.us.
Total runtime was 2m5s = 1.9ms per lookup.
It returned 16300 maxmind entries also found in blackholes.us, but also
49931 not found. I think the staleness of blackholes.us has been
confirmed.
/Per Jessen, Zürich
Re: blackholes.us ?
Posted by Matt Kettler <mk...@verizon.net>.
Per Jessen wrote:
> Matt Kettler wrote:
>
>
>> Per Jessen wrote:
>>
>>> It doesn't as far as the rejected mail goes - but it does matter wrt
>>> the usefulness of blackholes.us. Which is all I'm asking about.
>>>
>>>
>> True. Although this is the SpamAssassin mailing list, so, within
>> context of scored systems, the blackholes.us might be of some use.
>> That said, if you're using SA, it's much lighter weight to use the
>> RelayCountries plugin.
>>
>
> That seems hard to believe - lighter than a single local DNS query?
>
Single local query of a database. Not everyone using blackholes.us is
doing zone transfers, so for some folks it's heavy-weight because it's
an off-site DNS query.
When compared to someone using zone transfers, it's still likely to be
marginally faster because the Ip::Country::Fast code and database are
both optimized for doing this kind of query very quickly.
>> However, I will agree with you that blackholes.us is oft abused by
>> brain-dead admins who simply think "I don't know anyone in Germany, so
>> I'm going to blacklist it"...
>>
>
> It's the first time I've come across it, but it does seem a pretty dumb
> thing to do.
>
>
>> You also get the same effect with European admins blacklisting whole
>> major US ISP's (ie: verizon.net, comcast.net, etc).
>>
>
> Yep. Equally dumb.
>
>
>>> I'm not. I'm only concerned about the apparent staleness of the
>>> blackholes.us data. Does anyone have any info on that?
>>>
>> Nope. But I'd be more interested in information that the staleness was
>> resulting in inaccuracy.
>>
>
> That's easily checked - we could run a comparison of any up-to-date
> geoip database against blackholes.us.
>
>
True.
Re: blackholes.us ?
Posted by Per Jessen <pe...@computer.org>.
Matt Kettler wrote:
> Per Jessen wrote:
>> It doesn't as far as the rejected mail goes - but it does matter wrt
>> the usefulness of blackholes.us. Which is all I'm asking about.
>>
> True. Although this is the SpamAssassin mailing list, so, within
> context of scored systems, the blackholes.us might be of some use.
> That said, if you're using SA, it's much lighter weight to use the
> RelayCountries plugin.
That seems hard to believe - lighter than a single local DNS query?
> However, I will agree with you that blackholes.us is oft abused by
> brain-dead admins who simply think "I don't know anyone in Germany, so
> I'm going to blacklist it"...
It's the first time I've come across it, but it does seem a pretty dumb
thing to do.
> You also get the same effect with European admins blacklisting whole
> major US ISP's (ie: verizon.net, comcast.net, etc).
Yep. Equally dumb.
>> I'm not. I'm only concerned about the apparent staleness of the
>> blackholes.us data. Does anyone have any info on that?
>
> Nope. But I'd be more interested in information that the staleness was
> resulting in inaccuracy.
That's easily checked - we could run a comparison of any up-to-date
geoip database against blackholes.us.
/Per Jessen, Zürich
Re: blackholes.us ?
Posted by Andrzej Adam Filip <an...@xl.wp.pl>.
Matt Kettler wrote:
> [...]
> You also get the same effect with European admins blacklisting whole
> major US ISP's (ie: verizon.net, comcast.net, etc).
Verizon had blocked Europe.
The revenge is sweet :-)
http://www.google.com/search?q=verizon+blocking+europe
http://www.theregister.co.uk/2005/01/14/verizon_email_block/
Verizon persists with European email blockade
Unsplendid isolation By John Leyden
Published Friday 14th January 2005 16:17 GMT
> Both result from a failure to recognize that high spam volume isn't
> always due to being a spam domain..It could just be a really large
> number of people there, and if 1% of them have bots, that's a lot of
> spam. A small ISP or country with 50% infection rate would be much less
> noticeable. But then again, the nature of statisics and distributions is
> usually way beyond your average "MCSE in a week" boot-camp graduate.
I personally would suggest using "generic RDNS" blocks specially
tailored for such ISP.
--
[pl>en: Andrew] Andrzej Adam Filip : anfi@priv.onet.pl : anfi@xl.wp.pl
Re: blackholes.us ?
Posted by Matt Kettler <mk...@verizon.net>.
Per Jessen wrote:
> John Rudd wrote:
>
>
>>> The server is in Germany - anyway, I can't be bothered to deal with a
>>> mailadmin who rejects based on blackholes.us ... I just curious given
>>> the staleness of the data.
>>>
>> If the server is in Germany, then I don't see how staleness of the
>> data matters.
>>
>
> It doesn't as far as the rejected mail goes - but it does matter wrt the
> usefulness of blackholes.us. Which is all I'm asking about.
>
True. Although this is the SpamAssassin mailing list, so, within context
of scored systems, the blackholes.us might be of some use. That said, if
you're using SA, it's much lighter weight to use the RelayCountries plugin.
blackholes.us also has some application in sites that are by law
prohibited from exchanging data with certain countries (ie: military
contractors). Of course, such sites should be using more accurate and
fresh data. ie: maxmind's geoip services.
However, I will agree with you that blackholes.us is oft abused by
brain-dead admins who simply think "I don't know anyone in Germany, so
I'm going to blacklist it"...
You also get the same effect with European admins blacklisting whole
major US ISP's (ie: verizon.net, comcast.net, etc).
Both result from a failure to recognize that high spam volume isn't
always due to being a spam domain..It could just be a really large
number of people there, and if 1% of them have bots, that's a lot of
spam. A small ISP or country with 50% infection rate would be much less
noticeable. But then again, the nature of statisics and distributions is
usually way beyond your average "MCSE in a week" boot-camp graduate.
>> For two:
>> The mapping of IP to country doesn't change often. Thus, the data is
>> likely to appear to be "stale" just because it doesn't need to be
>> changed frequently (if at all).
>>
>
> None of the data at blackholes.us have changed for 18 months.
>
Hmm, that's sad. They might not have run any imports lately.. It's kinda
a small volunteer project AFAIK.
\
>> So, while you may feel like you can't be bothered to deal with a
>> mailadmin who rejects mail based on the sender being in Germany,
>> that's kind of what you're left with: if you're trying to fix this,
>>
>
> I'm not. I'm only concerned about the apparent staleness of the
> blackholes.us data. Does anyone have any info on that?
Nope. But I'd be more interested in information that the staleness was
resulting in inaccuracy.
Re: blackholes.us ?
Posted by Per Jessen <pe...@computer.org>.
John Rudd wrote:
>> The server is in Germany - anyway, I can't be bothered to deal with a
>> mailadmin who rejects based on blackholes.us ... I just curious given
>> the staleness of the data.
>
> If the server is in Germany, then I don't see how staleness of the
> data matters.
It doesn't as far as the rejected mail goes - but it does matter wrt the
usefulness of blackholes.us. Which is all I'm asking about.
> For two:
> The mapping of IP to country doesn't change often. Thus, the data is
> likely to appear to be "stale" just because it doesn't need to be
> changed frequently (if at all).
None of the data at blackholes.us have changed for 18 months.
> So, while you may feel like you can't be bothered to deal with a
> mailadmin who rejects mail based on the sender being in Germany,
> that's kind of what you're left with: if you're trying to fix this,
I'm not. I'm only concerned about the apparent staleness of the
blackholes.us data. Does anyone have any info on that?
/Per Jessen, Zürich
Re: blackholes.us ?
Posted by John Rudd <jr...@ucsc.edu>.
Per Jessen wrote:
> John D. Hardin wrote:
>
>> On Mon, 17 Dec 2007, Per Jessen wrote:
>>
>>> Does anyone have a current status for blackholes.us ? The rsync'ed
>>> data is about 18months old.
>>>
>>> I had an email rejected earlier today due to a server
>>> being "blacklisted" by germany.blackholes.us ....
>> Well, if the MTA is in Germany, the DNSBL isn't the problem.
>
> The server is in Germany - anyway, I can't be bothered to deal with a
> mailadmin who rejects based on blackholes.us ... I just curious given
> the staleness of the data.
If the server is in Germany, then I don't see how staleness of the data
matters.
For one:
(country).blackholes.us is a listing of every known address within a
given country (as opposed to it being a list about open proxies, virus
infected servers, spam senders, etc.; it's a location based list not an
abusive behavior based list). The server is in Germany, therefore they
belong on the Germany list.
For two:
The mapping of IP to country doesn't change often. Thus, the data is
likely to appear to be "stale" just because it doesn't need to be
changed frequently (if at all).
So, while you may feel like you can't be bothered to deal with a
mailadmin who rejects mail based on the sender being in Germany, that's
kind of what you're left with: if you're trying to fix this, then you
have to deal with that mailadmin. Otherwise, you just have to recognize
that they don't want email from Germany. Trying to get off of
germany.blackholes.us would require moving your server out of Germany,
or at least getting it an IP that isn't allocated to Germany.
Re: blackholes.us ?
Posted by Per Jessen <pe...@computer.org>.
John D. Hardin wrote:
> On Mon, 17 Dec 2007, Per Jessen wrote:
>
>> Does anyone have a current status for blackholes.us ? The rsync'ed
>> data is about 18months old.
>>
>> I had an email rejected earlier today due to a server
>> being "blacklisted" by germany.blackholes.us ....
>
> Well, if the MTA is in Germany, the DNSBL isn't the problem.
The server is in Germany - anyway, I can't be bothered to deal with a
mailadmin who rejects based on blackholes.us ... I just curious given
the staleness of the data.
/Per Jessen, Zürich
Re: blackholes.us ?
Posted by "John D. Hardin" <jh...@impsec.org>.
On Mon, 17 Dec 2007, Per Jessen wrote:
> Does anyone have a current status for blackholes.us ? The rsync'ed data
> is about 18months old.
>
> I had an email rejected earlier today due to a server
> being "blacklisted" by germany.blackholes.us ....
Well, if the MTA is in Germany, the DNSBL isn't the problem.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
"Bother," said Pooh as he struggled with /etc/sendmail.cf, "it never
does quite what I want. I wish Christopher Robin was here."
-- Peter da Silva in a.s.r
-----------------------------------------------------------------------
8 days until Christmas