You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Brijesh Deo <bd...@SonicWALL.com> on 2009/08/17 12:03:45 UTC
Re: MaxTotalConnections and MaxConnectionPerRoute in case ofhttpclient instance per target host
Hi Oleg,
Thanks for the reply.
Regarding the following point that you have mentioned, here is what I am
trying to do.
I personally see no benefit of having multiple HttpClient /
ConnectionManager,
(a pair per target host). A single HttpClient instance with
ThreadSafeClientConnManager would do the job just as well.
[Brijesh] Whenever a client (browser) requests access to any backend
http server through my reverse proxy, I have to perform an automatic
form based login into the backend server and then fetch the default page
and return the content in the HttpResponse to the client (browser) from
my Reverse proxy server. To perform an automatic form based login, I
would have all the necessary parameters like login params, username,
password, login post uri etc. stored beforehand for every backend server
host. Then I need to use the same session with the backend server to
execute further requests to the backend server that would come from the
client(browser).
Since I need to do such a thing with many backend servers, do you think
it is possible to do with a Single HttpClient instance? Can a single
HttpClient instance maintain login sessions with multiple backend
servers at the same time?
Also, other parameters like Timeout periods, cookie policy etc. could
vary from host to host.
These were the reasons that prompted me to go with a one HttpClient per
host approach to make things cleaner and less complex.
Do you think I could still make it work with a single http client
instance?
Please advise what you think.
Thanks,
Brijesh
-----Original Message-----
From: Oleg Kalnichevski [mailto:olegk@apache.org]
Sent: Monday, August 17, 2009 3:08 PM
To: HttpClient User Discussion
Subject: [Junk released by Allow List] Re: MaxTotalConnections and
MaxConnectionPerRoute in case ofhttpclient instance per target host
On Mon, Aug 17, 2009 at 12:37:37AM +0530, Brijesh Deo wrote:
> Hi,
>
>
>
> I am implementing an http reverse proxy, somewhat on the lines of the
> sample implementation - ElementalReverseProxy with some modifications.
I
> am using HttpCore classes to implement the http server (using
> org.apache.http.protocol.HttpService) and the HttpClient api (instead
of
> the HttpRequestExecutor) to implement the client side for sending the
> Http requests to the target servers and receiving Http Responses from
> them. I am using the latest HttpClient 4.0 binaries bundle (with
> dependencies).
>
>
>
> I need to provide access to several backend http web-servers through
my
> reverse proxy. For that, I am creating one HttpClient instance per
> backend host and all requests targeted to a particular target host are
> executed using the corresponding httpClient instance. Every httpClient
> instance uses separate instances of ThreadSafeClientConnManager. Every
> HttpClient-ThreadSafeClientConnManager pair has MaxTotalConnections =
20
> and MaxConnectionsPerRoute = 10.
>
>
I personally see no benefit of having multiple HttpClient /
ConnectionManager,
(a pair per target host). A single HttpClient instance with
ThreadSafeClientConnManager would do the job just as well.
>
> With respect to this I have the following questions:
>
>
>
> 1) Since I have one HttpClient-ThreadSafeClientConnManager instances
> pair for one target host, can there be multiple routes (HttpRoute) to
> the same target server while executing requests from the HttpClient?
Or
> would there be only one route to a given target host in this case?
>
>
Only one route, provide you do not use different proxies to access the
same
target host.
>
> 2) I have noticed that when multiple threads are executing requests to
a
> given target server using the corresponding HttpClient instance, I get
> ConnectionPoolTimeoutException at times when the no of connections
> touches MaxConnectionsPerRoute (10) even though the
MaxTotalConnections
> is 20. So, do all the connections use the same route in such
scenarios?
Apparently, they do
> Then, in cases like these MaxConnectionsPerRoute becomes the limiting
> value for the number of allowed connections rather than the
> MaxTotalConnections. Is this observation correct?
>
>
Yes, it is
>
> 3) In cases like this, do the two parameters MaxTotalConnections and
> MaxConnectionPerRoute need to have the same value?
>
>
No, they do not, but the exact setup depends on particular needs of your
applications.
For instance one many want to ensure that there is no more than 10
connections
to Host A and no more than 10 connections to Host B, but simultameously
there
is no more than 15 connections in total at the same time.
Oleg
>
> Please help in understanding/ validating this assumption.
>
>
>
> Thanks,
>
> Brijesh
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: MaxTotalConnections and MaxConnectionPerRoute in case
ofhttpclient instance per target host
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Mon, Aug 17, 2009 at 03:33:45PM +0530, Brijesh Deo wrote:
> Hi Oleg,
>
> Thanks for the reply.
> Regarding the following point that you have mentioned, here is what I am
> trying to do.
>
> I personally see no benefit of having multiple HttpClient /
> ConnectionManager,
> (a pair per target host). A single HttpClient instance with
> ThreadSafeClientConnManager would do the job just as well.
>
>
> [Brijesh] Whenever a client (browser) requests access to any backend
> http server through my reverse proxy, I have to perform an automatic
> form based login into the backend server and then fetch the default page
> and return the content in the HttpResponse to the client (browser) from
> my Reverse proxy server. To perform an automatic form based login, I
> would have all the necessary parameters like login params, username,
> password, login post uri etc. stored beforehand for every backend server
> host. Then I need to use the same session with the backend server to
> execute further requests to the backend server that would come from the
> client(browser).
>
> Since I need to do such a thing with many backend servers, do you think
> it is possible to do with a Single HttpClient instance? Can a single
> HttpClient instance maintain login sessions with multiple backend
> servers at the same time?
HTTP state is maintained based on the content of HTTP messages (usually using
cookies). With a few exceptions HTTP connections are usually state-less.
> Also, other parameters like Timeout periods, cookie policy etc. could
> vary from host to host.
> These were the reasons that prompted me to go with a one HttpClient per
> host approach to make things cleaner and less complex.
>
HTTP parameters can also be set on a per request basis. One does not have to
have different HttpClient instances just to maintain a different set of HTTP
parameters.
> Do you think I could still make it work with a single http client
> instance?
I believe so. You could also use multiple HttpClient instances sharing a single
connection manager instance.
Oleg
> Please advise what you think.
>
> Thanks,
> Brijesh
>
>
>
> -----Original Message-----
> From: Oleg Kalnichevski [mailto:olegk@apache.org]
> Sent: Monday, August 17, 2009 3:08 PM
> To: HttpClient User Discussion
> Subject: [Junk released by Allow List] Re: MaxTotalConnections and
> MaxConnectionPerRoute in case ofhttpclient instance per target host
>
> On Mon, Aug 17, 2009 at 12:37:37AM +0530, Brijesh Deo wrote:
> > Hi,
> >
> >
> >
> > I am implementing an http reverse proxy, somewhat on the lines of the
> > sample implementation - ElementalReverseProxy with some modifications.
> I
> > am using HttpCore classes to implement the http server (using
> > org.apache.http.protocol.HttpService) and the HttpClient api (instead
> of
> > the HttpRequestExecutor) to implement the client side for sending the
> > Http requests to the target servers and receiving Http Responses from
> > them. I am using the latest HttpClient 4.0 binaries bundle (with
> > dependencies).
> >
> >
> >
> > I need to provide access to several backend http web-servers through
> my
> > reverse proxy. For that, I am creating one HttpClient instance per
> > backend host and all requests targeted to a particular target host are
> > executed using the corresponding httpClient instance. Every httpClient
> > instance uses separate instances of ThreadSafeClientConnManager. Every
> > HttpClient-ThreadSafeClientConnManager pair has MaxTotalConnections =
> 20
> > and MaxConnectionsPerRoute = 10.
> >
> >
>
> I personally see no benefit of having multiple HttpClient /
> ConnectionManager,
> (a pair per target host). A single HttpClient instance with
> ThreadSafeClientConnManager would do the job just as well.
>
>
> >
> > With respect to this I have the following questions:
> >
> >
> >
> > 1) Since I have one HttpClient-ThreadSafeClientConnManager instances
> > pair for one target host, can there be multiple routes (HttpRoute) to
> > the same target server while executing requests from the HttpClient?
> Or
> > would there be only one route to a given target host in this case?
> >
> >
>
> Only one route, provide you do not use different proxies to access the
> same
> target host.
>
>
> >
> > 2) I have noticed that when multiple threads are executing requests to
> a
> > given target server using the corresponding HttpClient instance, I get
> > ConnectionPoolTimeoutException at times when the no of connections
> > touches MaxConnectionsPerRoute (10) even though the
> MaxTotalConnections
> > is 20. So, do all the connections use the same route in such
> scenarios?
>
> Apparently, they do
>
> > Then, in cases like these MaxConnectionsPerRoute becomes the limiting
> > value for the number of allowed connections rather than the
> > MaxTotalConnections. Is this observation correct?
> >
> >
>
> Yes, it is
>
>
> >
> > 3) In cases like this, do the two parameters MaxTotalConnections and
> > MaxConnectionPerRoute need to have the same value?
> >
> >
>
> No, they do not, but the exact setup depends on particular needs of your
> applications.
>
> For instance one many want to ensure that there is no more than 10
> connections
> to Host A and no more than 10 connections to Host B, but simultameously
> there
> is no more than 15 connections in total at the same time.
>
> Oleg
>
> >
> > Please help in understanding/ validating this assumption.
> >
> >
> >
> > Thanks,
> >
> > Brijesh
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org