You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Giampaolo Tomassoni <g....@libero.it> on 2007/09/26 15:11:20 UTC
URIWhois-0.02
Dears,
I got some feedback from people failing to successfully run the URIWhois
plugin.
It seems to me these are installation issues, so I just prepared version
0.02 of this plugin, which basically borrows an INSTALL.txt file in which I
attempt to explain how to correctly install the plugin. Also, the sample
URIWhois.cf shows an example of the "uri_whois_cache_path" setting, which is
meant to define where you want to put the cache database.
Please note that Jeff Chan is basically right in his reply
(<11...@mail.supranet.net>): the issue covered by the
URIWhois plugin would be much more efficiently solved by a centralized
solution, in which "someone" gathers registration data from registars (maybe
even not through whois, but through direct db access) and then publics this
data by some network-savvy protocol (like DNS, in example).
This is, however, something I see very far from being concretized: there is
a gTLD in every and each nation and the "central solver" would need to
gather consensus from all of them in order to obtain access to their DBs.
In the meantime, why not use a do-it-yourself way? Please note, however,
that this plugin is meant to help people with low mail exchanges. High
volumes may result in the counter-effects Jeff points out.
So, actually large providers better stay, well, at large from URIWhois?
Regards,
Giampaolo
R: URIWhois-0.02
Posted by Giampaolo Tomassoni <g....@libero.it>.
> -----Messaggio originale-----
> Da: Robert - elists [mailto:lists07@abbacomm.net]
>
>
> >
> > DOB, for example, is run by ar.com, who are a registrar. Since they
> are a
> > domain registrar, they have full, direct access to the whois
> database.
> >
> > Jeff C.
> >
>
> Well there ya go Jeff...
>
> Become a registrar and bam! More data to help you cause
When you guys say that "you just need to became a registrar and bah!", you
mean you get access to the whole database (even for gTLDs) or only to the
one for which you are a registrar?
I doubt you get the whole data. I may be wrong, anyway...
Giampaolo
>
> - rh
Re: R: R: R: URIWhois-0.02
Posted by Dave Koontz <dk...@mbc.edu>.
Thanks for the explanation, but I think you are missing the point here.
What is reasonable and what will cause a block?
An individual may well issue 100 queries a day for research. Not many
have the time to do tens of thousands+ a day (or more). Any system that
does will likely fall into the AUTOMATED clause. If your script/pm file
can not exit quickly on lookup failure, you could be stacking up
delivery queues on servers everywhere.
Giampaolo Tomassoni wrote:
>
> My rate is more or less 500 per day. Most is spam, which comes in more
> copies (for free!), or regular mail with no URIs, or URIs in signature but
> "common contacts" (more mails per day with the very same URIs). Thereby
> yesterday I had to issue only 100 queries over a volume of 500 mails. Today
> I expect more or less the same volume, but much less queries (I would bet on
> 40).
>
> Giampaolo
>
>
>
R: R: R: URIWhois-0.02
Posted by Giampaolo Tomassoni <g....@libero.it>.
> -----Messaggio originale-----
> Da: Dave Koontz [mailto:dkoontz@mbc.edu]
> Inviato: giovedì 27 settembre 2007 0.50
>
> If nothing else, you should likely add a disclaimer to your rules as
> you
> can't control the threshold at which a site may be blocked for
> excessive
> queries. I doubt that most users on this list have email volumes as
> low
> as yours (100?), and will go well above the thresholds you've tested.
> I
> am a what I consider a small site, yet I know I would generate well in
> excess of 100,000 queries in 24 hours.
My rate is more or less 500 per day. Most is spam, which comes in more
copies (for free!), or regular mail with no URIs, or URIs in signature but
"common contacts" (more mails per day with the very same URIs). Thereby
yesterday I had to issue only 100 queries over a volume of 500 mails. Today
I expect more or less the same volume, but much less queries (I would bet on
40).
Giampaolo
>
> So, what happens with your plugin should a timeout or ban occur?
A negative reply is saved in the cache. The negative reply entries expire by
default in one day (this can be tweaked), while positive ones in a month
(this too can be tweaked). Rules tied to whois queries are not run on a
negative reply. Since I'm only interested in the domain's creation date and
defined nameservers, positive replies could even last longer. Not too
longer, however, to avoid incurring in FPs due to the PARTNSMIS/FULLNSMIS
rule, should a domain change its nameservers.
Giampaolo
> Giampaolo Tomassoni wrote:
> > It depends upon how many e-mail you scan. In about 24h I just issued
> more or
> > less 100 queries to the to several TLDs' whois servers. What is it,
> 10
> > queries per TLD? It doesn't seem too much to me... Also note today
> I'm
> > probably not going to get the same numbers, thanks to caching.
> >
> > Of course, people scanning 1,000,000 messages a day would probably
> get
> > banned: it would end in roughly 200,000 queries issued per day (say
> 8,000
> > per TLD).
> >
> >
Re: R: R: URIWhois-0.02
Posted by Dave Koontz <dk...@mbc.edu>.
If nothing else, you should likely add a disclaimer to your rules as you
can't control the threshold at which a site may be blocked for excessive
queries. I doubt that most users on this list have email volumes as low
as yours (100?), and will go well above the thresholds you've tested. I
am a what I consider a small site, yet I know I would generate well in
excess of 100,000 queries in 24 hours.
So, what happens with your plugin should a timeout or ban occur?
Giampaolo Tomassoni wrote:
> It depends upon how many e-mail you scan. In about 24h I just issued more or
> less 100 queries to the to several TLDs' whois servers. What is it, 10
> queries per TLD? It doesn't seem too much to me... Also note today I'm
> probably not going to get the same numbers, thanks to caching.
>
> Of course, people scanning 1,000,000 messages a day would probably get
> banned: it would end in roughly 200,000 queries issued per day (say 8,000
> per TLD).
>
>
Re: R: R: URIWhois-0.02
Posted by Jeff Chan <je...@surbl.org>.
Quoting Giampaolo Tomassoni <g....@libero.it>:
> I think there is a lot of people in this list who runs a small business like
> mine, and who may benefit from using the URIWhois plugin with no negative
> consequences. The others, well, they have influence and resources to spend
> in a "centralized solution". I can't afford it.
SURBLs are free. Are you using them?
Jeff C.
Re: R: R: URIWhois-0.02
Posted by SM <sm...@resistor.net>.
At 15:32 26-09-2007, Giampaolo Tomassoni wrote:
>It depends upon how many e-mail you scan. In about 24h I just issued more or
>less 100 queries to the to several TLDs' whois servers. What is it, 10
>queries per TLD? It doesn't seem too much to me... Also note today I'm
>probably not going to get the same numbers, thanks to caching.
10 queries per TLD is not much when viewed on its own. It all
depends on the query rate and how many people are doing queries.
>I think there is a lot of people in this list who runs a small business like
>mine, and who may benefit from using the URIWhois plugin with no negative
>consequences. The others, well, they have influence and resources to spend
>in a "centralized solution". I can't afford it.
Thank you for the plugin. If you add a disclaimer, then nobody can
blame you for the consequences. :-)
Regards,
-sm
R: R: URIWhois-0.02
Posted by Giampaolo Tomassoni <g....@libero.it>.
> -----Messaggio originale-----
> Da: John D. Hardin [mailto:jhardin@impsec.org]
>
> On Wed, 26 Sep 2007, Giampaolo Tomassoni wrote:
>
> > So, I don't see the problem here: we are attempting to obtain
> > information about or related to a domain name registration record.
>
> Doing it over and over and over from an automated tool can be
> considered abusive when the service was intended to answer a
> relatively small volume of manually-initiated queries.
>
> Caching can reduce the impact, but it may not reduce it below the
> level considered abusive.
It depends upon how many e-mail you scan. In about 24h I just issued more or
less 100 queries to the to several TLDs' whois servers. What is it, 10
queries per TLD? It doesn't seem too much to me... Also note today I'm
probably not going to get the same numbers, thanks to caching.
Of course, people scanning 1,000,000 messages a day would probably get
banned: it would end in roughly 200,000 queries issued per day (say 8,000
per TLD).
Ok, fine. Mine is not the final weapon, but I don't see other ways apart
gathering data from every and each registrar delegated to maintain a given
TLD...
It seems to me that the main problem was due to the fact that ICANN
delegated domain operations *and* database maintenance to at least one org
per TLD, which means you have to gain access to at least 194 databases (no
hope they being in similar format or shipped by similar protocols), not to
mention .com, .net, .biz, etc. ones.
One can, of course, push ICANN to redesign the terms of its grants, but,
while this is not going to happen tomorrow, I would have a need for some
surrogate right now, you know...
I think there is a lot of people in this list who runs a small business like
mine, and who may benefit from using the URIWhois plugin with no negative
consequences. The others, well, they have influence and resources to spend
in a "centralized solution". I can't afford it.
Giampaolo
PS: John, how can I "catch" when an outstanding asynchronous query times out
in SA?
>
> --
> John Hardin KA7OHZ http://www.impsec.org/~jhardin/
> jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
> key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
> -----------------------------------------------------------------------
> Pelley: Will you pledge not to test a nuclear weapon?
> Ahmadeinejad: CIA! Secret prison in Europe! Abu Ghraib!
> -- Mahmoud Ahmadeinejad clumsily dodges a question
> (60 minutes interview, 9/20/2007)
> -----------------------------------------------------------------------
> 242 days until the Mars Phoenix lander arrives at Mars
Re: R: URIWhois-0.02
Posted by SM <sm...@resistor.net>.
At 13:48 26-09-2007, Giampaolo Tomassoni wrote:
>These terms depend upon the registrar offering the service.
>
>However, mostly they say that:
>
> "Whois database is provided by XXXXXX for information purposes only,
> and to assist persons in obtaining information about or related to a
> domain name registration record."
It also says:
"You are not authorized to access or query our Whois database
through the use
of electronic processes that are high-volume and automated except
as reasonably
necessary to register domain names or modify existing registrations"
>So, I don't see the problem here: we are attempting to obtain information
>about or related to a domain name registration record. We don't even care of
>contacts...
The problem is that this is an electronic process doing the queries
and it can be high-volume. Once we use a service for an unintended
use, it affects the accessibility of the service for normal
users. The rate limit set by the operator would kick in to prevent
"abusive" use.
Regards,
-sm
R: R: R: R: URIWhois-0.02
Posted by Giampaolo Tomassoni <g....@libero.it>.
> -----Messaggio originale-----
> Da: Michele Neylon :: Blacknight [mailto:michele@blacknight.ie]
> Inviato: giovedì 27 settembre 2007 0.55
>
> Giampaolo Tomassoni wrote:
> >> -----Messaggio originale-----
> >> Da: Michele Neylon :: Blacknight [mailto:michele@blacknight.ie]
> >>
> >> That refers to pretty much all TLDs and ccTLDs
> >>
> >> If you want to go slamming registry and registrar whois servers in
> an
> >> automated fashion you will get blacklisted by them all and blocked
> >
> > I don't want to slam anybody. I want just to get some information
> there is
> > actually no other way to obtain.
> >
> >
> >> Registrars don't use whois to check availability anyway ...
> >
> > I'm not checking domain availability.
>
> Maybe not, but you're displaying a lack of knowledge on whois data...
?!?!
> .es - no whois server
> .pl - no whois server
I guess a lot of spammers are going to buy a .es or .pl domain.
URIs from these places already get a penalty in my SA, anyway...
> .eu - no registrant data available over whois. rate limited
I'm not interested in the registrant data. You're displaying a lack of
knowledge on how URIWhois works.
> .ie - some data available. rate limited
>
>
> I could go on ...
No, please. Don't enumerate all the 194 gTLDs!
> Unless you are processing tiny amounts of mail you will get blocked and
> even if you don't you won't be able to retrieve much useful information
> in the first place
It depends by the TLD. Most give enough data for URIWhois to be useful. So,
why I and people with the same business size I have should not use it? For
the risk to get banned by some whois server? I believe they will not even
notice me...
Giampaolo
> --
> Mr Michele Neylon
> Blacknight Solutions
> Hosting & Colocation, Brand Protection
> http://www.blacknight.eu/
> http://blog.blacknight.eu/
> Tel. 1850 929 929
> Intl. +353 (0) 59 9183072
> Direct Dial: +353 (0)59 9183090
> Fax. +353 (0) 1 4811 763
> -------------------------------
> Blacknight Internet Solutions Ltd, Unit 12A,Barrowside Business
> Park,Sleaty Road,Graiguecullen,Carlow,Ireland Company No.: 370845
Re: R: R: R: URIWhois-0.02
Posted by "Michele Neylon :: Blacknight" <mi...@blacknight.ie>.
Giampaolo Tomassoni wrote:
>> -----Messaggio originale-----
>> Da: Michele Neylon :: Blacknight [mailto:michele@blacknight.ie]
>>
>> That refers to pretty much all TLDs and ccTLDs
>>
>> If you want to go slamming registry and registrar whois servers in an
>> automated fashion you will get blacklisted by them all and blocked
>
> I don't want to slam anybody. I want just to get some information there is
> actually no other way to obtain.
>
>
>> Registrars don't use whois to check availability anyway ...
>
> I'm not checking domain availability.
Maybe not, but you're displaying a lack of knowledge on whois data...
.es - no whois server
.pl - no whois server
.eu - no registrant data available over whois. rate limited
.ie - some data available. rate limited
I could go on ...
Unless you are processing tiny amounts of mail you will get blocked and
even if you don't you won't be able to retrieve much useful information
in the first place
--
Mr Michele Neylon
Blacknight Solutions
Hosting & Colocation, Brand Protection
http://www.blacknight.eu/
http://blog.blacknight.eu/
Tel. 1850 929 929
Intl. +353 (0) 59 9183072
Direct Dial: +353 (0)59 9183090
Fax. +353 (0) 1 4811 763
-------------------------------
Blacknight Internet Solutions Ltd, Unit 12A,Barrowside Business
Park,Sleaty Road,Graiguecullen,Carlow,Ireland Company No.: 370845
R: R: R: URIWhois-0.02
Posted by Giampaolo Tomassoni <g....@libero.it>.
> -----Messaggio originale-----
> Da: Michele Neylon :: Blacknight [mailto:michele@blacknight.ie]
>
> That refers to pretty much all TLDs and ccTLDs
>
> If you want to go slamming registry and registrar whois servers in an
> automated fashion you will get blacklisted by them all and blocked
I don't want to slam anybody. I want just to get some information there is
actually no other way to obtain.
> Registrars don't use whois to check availability anyway ...
I'm not checking domain availability.
> --
> Mr Michele Neylon
> Blacknight Solutions
> Hosting & Colocation, Brand Protection
> http://www.blacknight.eu/
> http://blog.blacknight.eu/
> Tel. 1850 929 929
> Intl. +353 (0) 59 9183072
> Direct Dial: +353 (0)59 9183090
> Fax. +353 (0) 1 4811 763
> -------------------------------
> Blacknight Internet Solutions Ltd, Unit 12A,Barrowside Business
> Park,Sleaty Road,Graiguecullen,Carlow,Ireland Company No.: 370845
Re: R: R: URIWhois-0.02
Posted by "Michele Neylon :: Blacknight" <mi...@blacknight.ie>.
Giampaolo Tomassoni wrote:
>
> This means nothing: what is a "high volume"? Also, you normally use the
> "whois" command. Isn't it an "automated, electronic process"? They are to
> say: "If we like to ban you, don't scream!".
>
> These terms comes from early internet ages, when spammers were used to
> scavenge their data from whois records too. They were meant to "scare"
> people abusing this service. Today, most domain registrars don't even public
> e-mail addresses anymore...
>
That refers to pretty much all TLDs and ccTLDs
If you want to go slamming registry and registrar whois servers in an
automated fashion you will get blacklisted by them all and blocked
Registrars don't use whois to check availability anyway ...
--
Mr Michele Neylon
Blacknight Solutions
Hosting & Colocation, Brand Protection
http://www.blacknight.eu/
http://blog.blacknight.eu/
Tel. 1850 929 929
Intl. +353 (0) 59 9183072
Direct Dial: +353 (0)59 9183090
Fax. +353 (0) 1 4811 763
-------------------------------
Blacknight Internet Solutions Ltd, Unit 12A,Barrowside Business
Park,Sleaty Road,Graiguecullen,Carlow,Ireland Company No.: 370845
R: R: R: R: URIWhois-0.02
Posted by Giampaolo Tomassoni <g....@libero.it>.
> -----Messaggio originale-----
> Da: Daryl C. W. O'Shea [mailto:spamassassin@dostech.ca]
> Inviato: giovedì 27 settembre 2007 19.22
>
> Giampaolo Tomassoni wrote:
>
> > Well, I have to manually turn my MX server on and off, sometimes. So,
> is it
> > SA an automated process? And then, if SA issues a whois request, is
> it
> > automated?
>
> This is plain silliness, which leads me to believe that you very well
> know that you're doing automated queries against whois servers against
> the wishes of the operators of those whois servers.
:)
>
>
> > It seems to me that the wording we see in whois replies is too loose
> to have
> > any meaning...
>
> The wording is clear as to how they intend the service to be used, and
> more importantly (and more clearly) how it is NOT to be used.
>
> Just because the wording is too vague to form a legal contract of any
> sort doesn't mean that you should ignore the wishes of the providers
> that it not be queried by an automated process.
I'm quite sure their intent is to avoid gathering data in bunches, which is
not what I do.
> > The problem, in fact, is that there is actually no other (easy) way
> than
> > whois to obtain the data you get by a whois query.
>
> The easiest way to obtain the amount of cash that I am interested in
> obtaining is to rob a bank. Perhaps I should do that, it is after all,
> the easy way. Who cares if it's right or wrong or against the wishes
> of
> the bank and its customers.
Now you are stating plain silliness: the two things are not comparable, not
even by law.
How can you damage a registrar by issuing whois queries? If you start
querying too much for what they regard as being too much, you'll get banned
and that's all...
> > The problem is that most
> > registrars do provide a whois interface not because they are tied by
> an
> > agreement to do so,
>
> Actually, that would be the number one reason they do so.
I don't think the terms of this agreement are too tight, then, since some
gTLD do avoid exposing any whois service (apart http forms)...
> > but rather because they know it is a useful tool which
> > effectively helps in daily registrar-to-registrar communication.
>
> As has been mentioned before, public whois services aren't used by
> registrars for domain management/etc.
Oh, come on: during the transfer process of domain X from registrar A to
registrar B, if B find some technical problem in the originating record, I
guess the first thing B does it to have a look at the whois record in the
A's server, in order to see if data matches.
This is what whois is basically meant to: an inter-registrar tool. Then, it
has the value of "deregulation", which was a key-word in 70's, and
"transparency", which was a key-word in late 80's. The latter 'till end
90's, when "privacy" took place.
> > At the end, the problem is in the terms by which ICANN delegates TLDs
> to
> > registrars, I believe.
>
> Regardless of any problems you see, the terms are as as they are.
> Intentionally abusing a service is not the way to go about getting the
> terms of use changed.
I don't see any abuse in randomly querying whois for some domains, provided
I'm not doing it to cause a DoS or using the replies to spam people. The
wording is meant to keep away DoS and bulk fetch of e-mail. You have to put
words in an historical context to get their meaning.
> >>> What is the source, by the way? Which TLD?
> >> Aside from the specific wording, that's pretty standard boilerplate
> for
> >> any registrar's WHOIS service that I've ever seen.
>
> Yeah, that one was from PIR, but they're all pretty much the same.
> Many
> are copied verbatim.
whois.nic.it, in example, doesn't even mention any term of usage.
Giampaolo
>
>
> Daryl
Re: R: R: R: URIWhois-0.02
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
Giampaolo Tomassoni wrote:
> Well, I have to manually turn my MX server on and off, sometimes. So, is it
> SA an automated process? And then, if SA issues a whois request, is it
> automated?
This is plain silliness, which leads me to believe that you very well
know that you're doing automated queries against whois servers against
the wishes of the operators of those whois servers.
> It seems to me that the wording we see in whois replies is too loose to have
> any meaning...
The wording is clear as to how they intend the service to be used, and
more importantly (and more clearly) how it is NOT to be used.
Just because the wording is too vague to form a legal contract of any
sort doesn't mean that you should ignore the wishes of the providers
that it not be queried by an automated process.
> The problem, in fact, is that there is actually no other (easy) way than
> whois to obtain the data you get by a whois query.
The easiest way to obtain the amount of cash that I am interested in
obtaining is to rob a bank. Perhaps I should do that, it is after all,
the easy way. Who cares if it's right or wrong or against the wishes of
the bank and its customers.
> The problem is that most
> registrars do provide a whois interface not because they are tied by an
> agreement to do so,
Actually, that would be the number one reason they do so.
> but rather because they know it is a useful tool which
> effectively helps in daily registrar-to-registrar communication.
As has been mentioned before, public whois services aren't used by
registrars for domain management/etc.
> At the end, the problem is in the terms by which ICANN delegates TLDs to
> registrars, I believe.
Regardless of any problems you see, the terms are as as they are.
Intentionally abusing a service is not the way to go about getting the
terms of use changed.
>>> What is the source, by the way? Which TLD?
>> Aside from the specific wording, that's pretty standard boilerplate for
>> any registrar's WHOIS service that I've ever seen.
Yeah, that one was from PIR, but they're all pretty much the same. Many
are copied verbatim.
Daryl
R: R: R: URIWhois-0.02
Posted by Giampaolo Tomassoni <g....@libero.it>.
> -----Messaggio originale-----
> Da: Kris Deugau [mailto:kdeugau@vianet.ca]
> Inviato: giovedì 27 settembre 2007 18.14
>
> Giampaolo Tomassoni wrote:
> >> -----Messaggio originale-----
> >> Da: Daryl C. W. O'Shea [mailto:spamassassin@dostech.ca]
> >> It appears that you selectively missed the part that applies:
> >>
> >> "You agree that you will use this data only for lawful purposes and
> >> that, under no circumstances will you use this data to: (a) [...] or
> >> (b)
> >> enable high volume, automated, electronic processes that send
> queries
> >> or
> >> data to the systems of Registry Operator or any ICANN-Accredited
> >> Registrar, except as reasonably necessary to register domain names
> or
> >> modify existing registrations."
> >
> > This means nothing: what is a "high volume"?
>
> Anything more than a few queries per minute (such as you might generate
> by doing lookups one by one, by hand, with a command-line WHOIS client,
> or via their web interface).
>
> > Also, you normally use the
> > "whois" command. Isn't it an "automated, electronic process"?
>
> That depends on how it's called. If you run it from the command line,
> by hand, it's not automated. If you call it in a loop from a script
> (or, oh, say, something like SA), then it's automated.
Well, I have to manually turn my MX server on and off, sometimes. So, is it
SA an automated process? And then, if SA issues a whois request, is it
automated?
It seems to me that the wording we see in whois replies is too loose to have
any meaning...
> > These terms comes from early internet ages, when spammers were used
> to
> > scavenge their data from whois records too. They were meant to
> "scare"
> > people abusing this service. Today, most domain registrars don't even
> public
> > e-mail addresses anymore...
>
> Mmmh. Sort of. The WHOIS system really *isn't* designed for large
> volumes of queries the way DNS is - why do you think sa-update or
> freshclam use DNS to tell the caller whether there's an update or not?
> Why are RBLs made available in DNS form rather than distributing a bare
> list of URIs or IP addresses via HTTP?
The problem, in fact, is that there is actually no other (easy) way than
whois to obtain the data you get by a whois query. The problem is that most
registrars do provide a whois interface not because they are tied by an
agreement to do so, but rather because they know it is a useful tool which
effectively helps in daily registrar-to-registrar communication.
At the end, the problem is in the terms by which ICANN delegates TLDs to
registrars, I believe.
> > What is the source, by the way? Which TLD?
>
> Aside from the specific wording, that's pretty standard boilerplate for
> any registrar's WHOIS service that I've ever seen.
>
> -kgd
Re: R: R: URIWhois-0.02
Posted by Kris Deugau <kd...@vianet.ca>.
Giampaolo Tomassoni wrote:
>> -----Messaggio originale-----
>> Da: Daryl C. W. O'Shea [mailto:spamassassin@dostech.ca]
>> It appears that you selectively missed the part that applies:
>>
>> "You agree that you will use this data only for lawful purposes and
>> that, under no circumstances will you use this data to: (a) [...] or
>> (b)
>> enable high volume, automated, electronic processes that send queries
>> or
>> data to the systems of Registry Operator or any ICANN-Accredited
>> Registrar, except as reasonably necessary to register domain names or
>> modify existing registrations."
>
> This means nothing: what is a "high volume"?
Anything more than a few queries per minute (such as you might generate
by doing lookups one by one, by hand, with a command-line WHOIS client,
or via their web interface).
> Also, you normally use the
> "whois" command. Isn't it an "automated, electronic process"?
That depends on how it's called. If you run it from the command line,
by hand, it's not automated. If you call it in a loop from a script
(or, oh, say, something like SA), then it's automated.
> These terms comes from early internet ages, when spammers were used to
> scavenge their data from whois records too. They were meant to "scare"
> people abusing this service. Today, most domain registrars don't even public
> e-mail addresses anymore...
Mmmh. Sort of. The WHOIS system really *isn't* designed for large
volumes of queries the way DNS is - why do you think sa-update or
freshclam use DNS to tell the caller whether there's an update or not?
Why are RBLs made available in DNS form rather than distributing a bare
list of URIs or IP addresses via HTTP?
> What is the source, by the way? Which TLD?
Aside from the specific wording, that's pretty standard boilerplate for
any registrar's WHOIS service that I've ever seen.
-kgd
R: R: URIWhois-0.02
Posted by Giampaolo Tomassoni <g....@libero.it>.
> -----Messaggio originale-----
> Da: Daryl C. W. O'Shea [mailto:spamassassin@dostech.ca]
> > However, mostly they say that:
> >
> > "Whois database is provided by XXXXXX for information purposes
> only,
> > and to assist persons in obtaining information about or related
> to a
> > domain name registration record."
> >
> > So, I don't see the problem here: we are attempting to obtain
> information
> > about or related to a domain name registration record. We don't even
> care of
> > contacts...
>
> It appears that you selectively missed the part that applies:
>
> "You agree that you will use this data only for lawful purposes and
> that, under no circumstances will you use this data to: (a) [...] or
> (b)
> enable high volume, automated, electronic processes that send queries
> or
> data to the systems of Registry Operator or any ICANN-Accredited
> Registrar, except as reasonably necessary to register domain names or
> modify existing registrations."
This means nothing: what is a "high volume"? Also, you normally use the
"whois" command. Isn't it an "automated, electronic process"? They are to
say: "If we like to ban you, don't scream!".
These terms comes from early internet ages, when spammers were used to
scavenge their data from whois records too. They were meant to "scare"
people abusing this service. Today, most domain registrars don't even public
e-mail addresses anymore...
What is the source, by the way? Which TLD?
Giampaolo
>
>
> Daryl
Re: R: URIWhois-0.02
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
Giampaolo Tomassoni wrote:
>> -----Messaggio originale-----
>> Da: SM [mailto:sm@resistor.net]
>>
>> At 08:56 26-09-2007, Robert - elists wrote:
>>> Become a registrar and bam! More data to help you cause
>> Access to Whois data is not limited to registrars only.
>>
>> I suggest reading the Whois terms of service before enabling this
>> plugin.
>
> These terms depend upon the registrar offering the service.
>
> However, mostly they say that:
>
> "Whois database is provided by XXXXXX for information purposes only,
> and to assist persons in obtaining information about or related to a
> domain name registration record."
>
> So, I don't see the problem here: we are attempting to obtain information
> about or related to a domain name registration record. We don't even care of
> contacts...
It appears that you selectively missed the part that applies:
"You agree that you will use this data only for lawful purposes and
that, under no circumstances will you use this data to: (a) [...] or (b)
enable high volume, automated, electronic processes that send queries or
data to the systems of Registry Operator or any ICANN-Accredited
Registrar, except as reasonably necessary to register domain names or
modify existing registrations."
Daryl
Re: R: URIWhois-0.02
Posted by "John D. Hardin" <jh...@impsec.org>.
On Wed, 26 Sep 2007, Giampaolo Tomassoni wrote:
> So, I don't see the problem here: we are attempting to obtain
> information about or related to a domain name registration record.
Doing it over and over and over from an automated tool can be
considered abusive when the service was intended to answer a
relatively small volume of manually-initiated queries.
Caching can reduce the impact, but it may not reduce it below the
level considered abusive.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Pelley: Will you pledge not to test a nuclear weapon?
Ahmadeinejad: CIA! Secret prison in Europe! Abu Ghraib!
-- Mahmoud Ahmadeinejad clumsily dodges a question
(60 minutes interview, 9/20/2007)
-----------------------------------------------------------------------
242 days until the Mars Phoenix lander arrives at Mars
R: URIWhois-0.02
Posted by Giampaolo Tomassoni <g....@libero.it>.
> -----Messaggio originale-----
> Da: SM [mailto:sm@resistor.net]
>
> At 08:56 26-09-2007, Robert - elists wrote:
> >Become a registrar and bam! More data to help you cause
>
> Access to Whois data is not limited to registrars only.
>
> I suggest reading the Whois terms of service before enabling this
> plugin.
These terms depend upon the registrar offering the service.
However, mostly they say that:
"Whois database is provided by XXXXXX for information purposes only,
and to assist persons in obtaining information about or related to a
domain name registration record."
So, I don't see the problem here: we are attempting to obtain information
about or related to a domain name registration record. We don't even care of
contacts...
Giampaolo
> Regards,
> -sm
RE: URIWhois-0.02
Posted by SM <sm...@resistor.net>.
At 08:56 26-09-2007, Robert - elists wrote:
>Become a registrar and bam! More data to help you cause
Access to Whois data is not limited to registrars only.
I suggest reading the Whois terms of service before enabling this plugin.
Regards,
-sm
RE: URIWhois-0.02
Posted by Robert - elists <li...@abbacomm.net>.
>
> Thats the easy answer, but do you know what it costs to become a
> registrar?
>
> Just for com/net from verisign you have $6500 up front, and $4k
> recurring. To get your icann credentials, you have $2500 up front with
> application, $4k yearly. A variable fee to icann once you start
> registering domains, and obviously the $0.25/registration that goes to
> icann. You also have to be able to show $70,000 in working capital.
>
> And that only gets you com and net. I'd want org from pir, info from
> afilias, and any other tld that takes alot of abuse (cn is big right
> now).
>
> At least those fees keep the bad guys from becoming registrars too.
>
> --
> Dallas Engelken
Dallas,
Yes, I do know the basic cost structures.
Hmmmmm, lemme see... money is not the largest concern if you know the owner
of the cattle upon a thousand hills.
Got a need, sow a seed.
- rh
Re: URIWhois-0.02
Posted by Dallas Engelken <da...@uribl.com>.
Robert - elists wrote:
>> DOB, for example, is run by ar.com, who are a registrar. Since they are a
>> domain registrar, they have full, direct access to the whois database.
>>
>> Jeff C.
>>
>>
>
> Well there ya go Jeff...
>
> Become a registrar and bam! More data to help you cause
>
>
Thats the easy answer, but do you know what it costs to become a registrar?
Just for com/net from verisign you have $6500 up front, and $4k
recurring. To get your icann credentials, you have $2500 up front with
application, $4k yearly. A variable fee to icann once you start
registering domains, and obviously the $0.25/registration that goes to
icann. You also have to be able to show $70,000 in working capital.
And that only gets you com and net. I'd want org from pir, info from
afilias, and any other tld that takes alot of abuse (cn is big right now).
At least those fees keep the bad guys from becoming registrars too.
--
Dallas Engelken
dallase@uribl.com
http://uribl.com
RE: URIWhois-0.02
Posted by Robert - elists <li...@abbacomm.net>.
>
> DOB, for example, is run by ar.com, who are a registrar. Since they are a
> domain registrar, they have full, direct access to the whois database.
>
> Jeff C.
>
Well there ya go Jeff...
Become a registrar and bam! More data to help you cause
- rh
Re: URIWhois-0.02
Posted by Jeff Chan <je...@surbl.org>.
Quoting Giampaolo Tomassoni <g....@libero.it>:
> the issue covered by the
> URIWhois plugin would be much more efficiently solved by a centralized
> solution, in which "someone" gathers registration data from registars (maybe
> even not through whois, but through direct db access) and then publics this
> data by some network-savvy protocol (like DNS, in example).
DOB, for example, is run by ar.com, who are a registrar. Since they are a
domain registrar, they have full, direct access to the whois database.
Jeff C.