If I use MultiThreadedHttpConnectionManager, will it cause issues with load balancers?

["KARR, DAVID (ATTSI)" <dk...@att.com>]

I'm investigating converting some code using HttpClient to use the
MultiThreadedHttpConnectionManager, which will allow some cached
connections to be able to skip the SSL handshake.  I was wondering if
there are any disadvantages of this approach.

If the URL I'm connecting to is a load balancer, I would think that
means that the connection that gets set up goes directly to a target
node, and not to the load balancer.  I would think a load balancer
either uses a "round-robin" or "load measurement" strategy, or some
combination of both.  If it primarily measures load to determine where
to bind a connection, then doesn't the presence of cached connections
make it more difficult for this kind of load balancer to balance load
effectively?

I noticed that HttpClient has some notion of expiring "idle"
connections, with the "closeIdleConnections()" on
MultiThreadedHttpConnectionManager.  I'm not sure what this does,
however.  From the wording of the javadoc, it makes it sound like it
performs the action when the method is called (closing idle
connections), as opposed to simply automatically closing all connections
that have been idle that long (which seems much more logical).  Which
interpretation is correct?

I'm not sure whether closing idle connections will mitigate the concern
I have about load balancing, but it might help.

I read the "performance guide", but I didn't see these issues addressed.

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org

Re: If I use MultiThreadedHttpConnectionManager, will it cause issues with load balancers?

[sa...@gmail.com]

Hi David,

I've got some experience in this space, so hopefully I can help.

Load balancers use a variety of policies to determine which backend to use. Typically you can configure "round robin", "least connections", "fastest", and many others too. Using persistent connections will avoid this selection, yes. However, in real world scenarios you'd typically have many connections going via the load balancer, and these would be distributed over the backends. So unless you have only one persistent connection through the LB and it hammers one particular backend as a result, it's not usually a problem in the real world.

The closing of idle connections is important, but not just for the reason above. Most load balancers will enforce a timeout on inactive connections, and they *may* drop such connections silently (I.e. You won't get a TCP RST or FIN, so you don't get any socket errors). When this occurs your client will try to re-use the dead connection and the request will fail.

I seem to remember there being two method calls in the IdleConnectionEvictor class that Oleg included in the documentation. These corresponded to the two interpretations you mentioned. I'll need to check the code though. We were definitely able to close only connections that had been idle for more than X seconds though.

Ultimately, using persistent connections has a few pain points, but it improves throughput considerably (especially with SSL).

Thanks,

Sam

Sent from my BlackBerry® wireless device

-----Original Message-----
From: "KARR, DAVID (ATTSI)" <dk...@att.com>
Date: Fri, 6 Aug 2010 09:19:07 
To: HttpClient User Discussion<ht...@hc.apache.org>
Reply-To: "HttpClient User Discussion" <ht...@hc.apache.org>
Subject: If I use MultiThreadedHttpConnectionManager, will it cause issues with load balancers?

I'm investigating converting some code using HttpClient to use the
MultiThreadedHttpConnectionManager, which will allow some cached
connections to be able to skip the SSL handshake.  I was wondering if
there are any disadvantages of this approach.

If the URL I'm connecting to is a load balancer, I would think that
means that the connection that gets set up goes directly to a target
node, and not to the load balancer.  I would think a load balancer
either uses a "round-robin" or "load measurement" strategy, or some
combination of both.  If it primarily measures load to determine where
to bind a connection, then doesn't the presence of cached connections
make it more difficult for this kind of load balancer to balance load
effectively?

I noticed that HttpClient has some notion of expiring "idle"
connections, with the "closeIdleConnections()" on
MultiThreadedHttpConnectionManager.  I'm not sure what this does,
however.  From the wording of the javadoc, it makes it sound like it
performs the action when the method is called (closing idle
connections), as opposed to simply automatically closing all connections
that have been idle that long (which seems much more logical).  Which
interpretation is correct?

I'm not sure whether closing idle connections will mitigate the concern
I have about load balancing, but it might help.

I read the "performance guide", but I didn't see these issues addressed.

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org