You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Sebastiano Vigna <vi...@di.unimi.it> on 2013/06/06 18:25:48 UTC
Forcing IP address in HTTP requests
It would be useful to be able to start an HTTP request towards a server specifying the IP address together with the request, so that no DNS resolution is performed (unless, of course, redirection handling is enabled).
This effect can be obtained with a custom DNS resolution class, but for a number of reasons (typically, to avoid double caching) we would find useful to have a direct way to do it.
Maybe there's a way and we missed it. Suggestions? Should I add a Jira issue?
seba
Ciao,
seba
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Forcing IP address in HTTP requests
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Thu, 2013-06-06 at 23:45 +0200, Sebastiano Vigna wrote:
> On 6 Jun 2013, at 8:56 PM, Oleg Kalnichevski <ol...@apache.org> wrote:
>
> > What's wrong with using an explicit HttpHost with a resolved address?
> > ---
> > CloseableHttpClient httpClient = HttpClients.createDefault();
> > HttpHost target = new HttpHost("173.194.40.20");
>
> Ahem... :) Of course. The problem is that somehow when looking for something like this I thought that HttpHost constructors requiring a String required the same string of the request.
>
> If I might make a suggestion, having a constructor accepting an InetAddress would make it very clear that this is not the case when browsing through the API.
>
Sure. Please raise a JIRA in HttpCore for this enhancement.
https://issues.apache.org/jira/browse/HTTPCORE
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Forcing IP address in HTTP requests
Posted by Sebastiano Vigna <vi...@di.unimi.it>.
On 6 Jun 2013, at 8:56 PM, Oleg Kalnichevski <ol...@apache.org> wrote:
> What's wrong with using an explicit HttpHost with a resolved address?
> ---
> CloseableHttpClient httpClient = HttpClients.createDefault();
> HttpHost target = new HttpHost("173.194.40.20");
Ahem... :) Of course. The problem is that somehow when looking for something like this I thought that HttpHost constructors requiring a String required the same string of the request.
If I might make a suggestion, having a constructor accepting an InetAddress would make it very clear that this is not the case when browsing through the API.
Thanks!
Ciao,
seba
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Forcing IP address in HTTP requests
Posted by Sebastiano Vigna <vi...@di.unimi.it>.
So sorry. Forget about it. Our university server have this crazy not-found page at users.unimi.it to which almost all robots.txt request are redirected.
Go figure.
Ciao,
seba
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Forcing IP address in HTTP requests
Posted by Sebastiano Vigna <vi...@di.unimi.it>.
On 18 Jun 2013, at 3:29 PM, Oleg Kalnichevski <ol...@apache.org> wrote:
> On Tue, 2013-06-18 at 15:23 +0200, Sebastiano Vigna wrote:
>> On 18 Jun 2013, at 3:20 PM, Oleg Kalnichevski <ol...@apache.org> wrote:
>>
>>> Are you using plain HttpCore or HttpClient?
>>
>> DefaultHttpClient, actually. We're moving soon to the new (non-deprecated) classes.
>>
>> Ciao,
>>
>
> Could you please run your app with connection management / context
> logging turned on and try to correlate what is going on inside
> HttpClient with DNS lookups?
We're using the new HttpClient now.
I have run the application and logged everything at DEBUG level. Moreover, I have the stack trace of every DNS call. We had a little run on our university server: all servers have 1 to 3 requests (OK because of redirects, which are handled by the client), but when fetching users.unimi.it DNS was resolved at each fetch. Can I email you the data?
Ciao,
seba
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Forcing IP address in HTTP requests
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2013-06-18 at 15:23 +0200, Sebastiano Vigna wrote:
> On 18 Jun 2013, at 3:20 PM, Oleg Kalnichevski <ol...@apache.org> wrote:
>
> > Are you using plain HttpCore or HttpClient?
>
> DefaultHttpClient, actually. We're moving soon to the new (non-deprecated) classes.
>
> Ciao,
>
Could you please run your app with connection management / context
logging turned on and try to correlate what is going on inside
HttpClient with DNS lookups?
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Forcing IP address in HTTP requests
Posted by Sebastiano Vigna <vi...@di.unimi.it>.
On 18 Jun 2013, at 3:20 PM, Oleg Kalnichevski <ol...@apache.org> wrote:
> Are you using plain HttpCore or HttpClient?
DefaultHttpClient, actually. We're moving soon to the new (non-deprecated) classes.
Ciao,
seba
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Forcing IP address in HTTP requests
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2013-06-18 at 15:11 +0200, Sebastiano Vigna wrote:
> On 6 Jun 2013, at 8:56 PM, Oleg Kalnichevski <ol...@apache.org> wrote:
>
> > On Thu, 2013-06-06 at 18:25 +0200, Sebastiano Vigna wrote:
> > What's wrong with using an explicit HttpHost with a resolved address?
>
>
> OK. We have implemented it that way (and thanks for adding the INetAddress constructor) but we're still seeing, for a few sites, several calls to DNS. We should see just redirected calls, but for some sites all requests go through the DNS resolver even if we pass an explicit address. We can try to isolate the problem inside HttpCore with some help, but for now we're frankly clueless.
>
> Does this ring a bell?
>
> Ciao,
>
Sebastiano
Are you using plain HttpCore or HttpClient?
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Forcing IP address in HTTP requests
Posted by Sebastiano Vigna <vi...@di.unimi.it>.
On 6 Jun 2013, at 8:56 PM, Oleg Kalnichevski <ol...@apache.org> wrote:
> On Thu, 2013-06-06 at 18:25 +0200, Sebastiano Vigna wrote:
> What's wrong with using an explicit HttpHost with a resolved address?
OK. We have implemented it that way (and thanks for adding the INetAddress constructor) but we're still seeing, for a few sites, several calls to DNS. We should see just redirected calls, but for some sites all requests go through the DNS resolver even if we pass an explicit address. We can try to isolate the problem inside HttpCore with some help, but for now we're frankly clueless.
Does this ring a bell?
Ciao,
seba
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Forcing IP address in HTTP requests
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Thu, 2013-06-06 at 18:25 +0200, Sebastiano Vigna wrote:
> It would be useful to be able to start an HTTP request towards a server specifying the IP address together with the request, so that no DNS resolution is performed (unless, of course, redirection handling is enabled).
>
> This effect can be obtained with a custom DNS resolution class, but for a number of reasons (typically, to avoid double caching) we would find useful to have a direct way to do it.
>
> Maybe there's a way and we missed it. Suggestions? Should I add a Jira issue?
>
What's wrong with using an explicit HttpHost with a resolved address?
---
CloseableHttpClient httpClient = HttpClients.createDefault();
HttpHost target = new HttpHost("173.194.40.20");
HttpGet httpGet = new HttpGet("http://www.google.com/");
CloseableHttpResponse httpResponse = httpClient.execute(
target, httpGet);
try {
System.out.println(httpResponse.getStatusLine());
} finally {
httpResponse.close();
}
---
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org