You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by mpindyala <mo...@rovicorp.com> on 2011/12/06 00:07:49 UTC
httpclient.defaultMaxPerRoute and maxTotalConnections questions
We have a java web application which makes calls to 2 of our third party
Services using httpclient.
Following are the settings for
org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManagerr. the numbers
are ridiculously high
httpclient.defaultMaxPerRoute = 10000
httpclient.maxTotalConnections = 20000
Socket and connection timeout on org.apache.http.params.HttpConnectionParams
are set as
httpclient.socketTimeout = 120000ms
httpclient.connectionTimeout = 20000ms
we have disabled the
org.apache.http.params.HttpConnectionParams#setStaleCheckingEnabled = false.
we have a cron process running which checks for idle connections using
ClientConnectionManager object every 10 seconds and closes expired & idle
connections.
We also have retry handler set on
org.apache.http.impl.client.DefaultHttpClient. It retries once with a 40ms
delay.
So questions are as follows
1. Does above setting look ok for you ? if not what are the optimal numbers.
Lets say there are 4 third party services we have to call , how should we
set these numbers to support that.
2. We sometimes get following errors, which are not able to figure out if it
is the third party server or httpclient issue. These errors show up during
peak volume hitting our servers.
a. java.net.SocketException: Connection reset
b. :java.net.SocketTimeoutException: Read timed out
c. org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for
connection
we checked with our third party services and everything was normal on their
side when these things happened.
Following are the stack traces for each exception type
org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for
connection
at
org.apache.http.impl.conn.tsccm.ConnPoolByRoute.getEntryBlocking(ConnPoolByRoute.java:417)
at
org.apache.http.impl.conn.tsccm.ConnPoolByRoute$1.getPoolEntry(ConnPoolByRoute.java:300)
at
org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager$1.getConnection(ThreadSafeClientConnManager.java:224)
at
gwy.core.io.TrackingClientConnectionRequest.getConnection(TrackingClientConnectionRequest.java:68)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:391)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:168)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149)
at
org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:110)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:260)
at
org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252)
at
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:281)
at
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
at
org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:219)
at
gwy.core.io.TrackingClientConnection.receiveResponseHeader(TrackingClientConnection.java:140)
at
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:622)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:454)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149)
at
org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:110)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:260)
at
org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252)
at
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:281)
at
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
at
org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:219)
at
gwy.core.io.TrackingClientConnection.receiveResponseHeader(TrackingClientConnection.java:140)
at
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:622)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:454)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
--
View this message in context: http://old.nabble.com/httpclient.defaultMaxPerRoute-and-maxTotalConnections-questions-tp32921044p32921044.html
Sent from the HttpClient-User mailing list archive at Nabble.com.
Re: httpclient.defaultMaxPerRoute and maxTotalConnections questions
Posted by William Speirs <ws...@apache.org>.
On Wed, Dec 7, 2011 at 6:39 PM, mpindyala <mo...@rovicorp.com> wrote:
> so you are saying max per route cannot be more than maxtotal connections?
Yes. My explanation before was confusing at best. As I understand it,
a route is to a specific host. A connection is just a socket. So you
can limit both the number of connections you make to a single host,
max per route, and the number of overall connections you make to *any*
host, max total connections.
If you make max per route > max total connections, then max per route
will essentially NOT be considered as max total connections will be
your limiting factor.
> Can I get your httpclient settings for achieving 65k per sec ? it would be really useful to us.
I didn't do anything special. I think I simply had 2000 total
connections and max per route (as I was going to the same host).
However, the host was on a LAN and had a very quick response.
Again, I would enable logging to see exactly what is going on. You
often think you're going down the same route, but aren't. Enabling
logging will show you when it's allocating a new request, and when
it's reusing an existing connection. This will kill your performance,
but at least you'll see how the client is behaving and make any
changes to your code if needed.
Bill-
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: httpclient.defaultMaxPerRoute and maxTotalConnections questions
Posted by mpindyala <mo...@rovicorp.com>.
William Speirs wrote:
>
> On Tue, Dec 6, 2011 at 1:30 PM, mpindyala <mo...@rovicorp.com>
> wrote:
>>> Yes , its just a thread . We did this to save 10-30ms for each http
>>> request that application does to back end providers. It seems connection
>>> manager takes 10-30ms just to close idle and expired connections before
>>> they can be reused. Please let us know if there are any pitfalls of
>>> doing
>>> this.
>
> I don't know if this makes sense... if the connection is idle it can
> be re-used for another request, why would you close it? You can
> however set the max amount of time the ThreadSafeClientConnManager
> will keep a connection open for:
> http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/conn/tsccm/ThreadSafeClientConnManager.html#ThreadSafeClientConnManager(org.apache.http.conn.scheme.SchemeRegistry,
> long, java.util.concurrent.TimeUnit)
>
> Thanks you Bill, I really appreciate your input.
>
> So we are closing idleconnections only if they are idle for longer than
> 10 seconds. we will retest our app with your suggestion and see the
> performance impact.
>
>>> The reason we set those numbers high as a protection, our third responds
>>> within 500ms most of the time.
>>> It is just when there is something going on between hops , the delay is
>>> seem.
>>> So one questions which still needs clarification is
>>> httpclient.defaultMaxPerRoute = 10000
>>> httpclient.maxTotalConnections = 20000
>>>
>>> is defaultMaxPerRoute is per third party service we are calling ? if
>>> yes
>>> then
>>> if there are 4 services we have to call then settings should be
>>> httpclient.defaultMaxPerRoute = 10000
>>> httpclient.maxTotalConnections = 40000
>>> Each route getting 10000 , is that assumption correct ?
>
> I would need Oleg to verify (or spend more time than I want checking
> the source), but these are different things. From the docs for
> ConnPoolByRoute
> (http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/conn/tsccm/ConnPoolByRoute.html),
> "connections are re-used only for the exact same route" & "connection
> limits are enforced per route rather than per host". So as I
> understand it a route and a host/connection are NOT the same thing. So
> you could have 4 as your MaxPerRoute and 1 as your
> maxTotalConnections... I don't think maxPerRoute >
> maxTotalConnections.
>
> so you are saying max per route cannot be more than maxtotal connections
> ?
>
>>> we want to support upto 4M per hour
>
> Depending upon a lot of factors 4M per hour should be doable... I've
> been able to get 65K per sec.
> Can I get your httpclient settings for achieving 65k per sec ? it would be
> really useful to us.
>
> Thank you
> Mohan
>
> Bill-
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
>
--
View this message in context: http://old.nabble.com/httpclient.defaultMaxPerRoute-and-maxTotalConnections-questions-tp32921044p32930550.html
Sent from the HttpClient-User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: httpclient.defaultMaxPerRoute and maxTotalConnections questions
Posted by William Speirs <ws...@apache.org>.
On Tue, Dec 6, 2011 at 1:30 PM, mpindyala <mo...@rovicorp.com> wrote:
>> Yes , its just a thread . We did this to save 10-30ms for each http
>> request that application does to back end providers. It seems connection
>> manager takes 10-30ms just to close idle and expired connections before
>> they can be reused. Please let us know if there are any pitfalls of doing
>> this.
I don't know if this makes sense... if the connection is idle it can
be re-used for another request, why would you close it? You can
however set the max amount of time the ThreadSafeClientConnManager
will keep a connection open for:
http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/conn/tsccm/ThreadSafeClientConnManager.html#ThreadSafeClientConnManager(org.apache.http.conn.scheme.SchemeRegistry,
long, java.util.concurrent.TimeUnit)
>> The reason we set those numbers high as a protection, our third responds
>> within 500ms most of the time.
>> It is just when there is something going on between hops , the delay is
>> seem.
>> So one questions which still needs clarification is
>> httpclient.defaultMaxPerRoute = 10000
>> httpclient.maxTotalConnections = 20000
>>
>> is defaultMaxPerRoute is per third party service we are calling ? if yes
>> then
>> if there are 4 services we have to call then settings should be
>> httpclient.defaultMaxPerRoute = 10000
>> httpclient.maxTotalConnections = 40000
>> Each route getting 10000 , is that assumption correct ?
I would need Oleg to verify (or spend more time than I want checking
the source), but these are different things. From the docs for
ConnPoolByRoute
(http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/conn/tsccm/ConnPoolByRoute.html),
"connections are re-used only for the exact same route" & "connection
limits are enforced per route rather than per host". So as I
understand it a route and a host/connection are NOT the same thing. So
you could have 4 as your MaxPerRoute and 1 as your
maxTotalConnections... I don't think maxPerRoute >
maxTotalConnections.
>> we want to support upto 4M per hour
Depending upon a lot of factors 4M per hour should be doable... I've
been able to get 65K per sec.
Bill-
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: httpclient.defaultMaxPerRoute and maxTotalConnections questions
Posted by mpindyala <mo...@rovicorp.com>.
William Speirs wrote:
>
> On Mon, Dec 5, 2011 at 6:07 PM, mpindyala <mo...@rovicorp.com>
> wrote:
>> We have a java web application which makes calls to 2 of our third party
>> Services using httpclient.
>>
>> Following are the settings for
>> org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManagerr. the numbers
>> are ridiculously high
>> httpclient.defaultMaxPerRoute = 10000
>> httpclient.maxTotalConnections = 20000
>>
>> Socket and connection timeout on
>> org.apache.http.params.HttpConnectionParams
>> are set as
>> httpclient.socketTimeout = 120000ms
>> httpclient.connectionTimeout = 20000ms
>>
>> we have disabled the
>> org.apache.http.params.HttpConnectionParams#setStaleCheckingEnabled =
>> false.
>> we have a cron process running which checks for idle connections using
>> ClientConnectionManager object every 10 seconds and closes expired & idle
>> connections.
>
> What do you mean a "cron process"? A thread that checks? Why would you
> do this at all? Why wouldn't you let the connection manager handle
> this for you?
>
> Yes , its just a thread . We did this to save 10-30ms for each http
> request that application does to back end providers. It seems connection
> manager takes 10-30ms just to close idle and expired connections before
> they can be reused. Please let us know if there are any pitfalls of doing
> this.
>
>> We also have retry handler set on
>> org.apache.http.impl.client.DefaultHttpClient. It retries once with a
>> 40ms
>> delay.
>>
>>
>> So questions are as follows
>> 1. Does above setting look ok for you ? if not what are the optimal
>> numbers.
>> Lets say there are 4 third party services we have to call , how should we
>> set these numbers to support that.
>
> There are no real "optimal numbers" it is more a matter of the service
> level agreement of the 3rd party and the expectations of your program.
> You have a socket timeout of 2 minutes and a connection timeout of 20
> seconds. Those seem high to me, but again, you might be doing
> something which necessitates this.
>
> The reason we set those numbers high as a protection, our third responds
> within 500ms most of the time.
> It is just when there is something going on between hops , the delay is
> seem.
> So one questions which still needs clarification is
> httpclient.defaultMaxPerRoute = 10000
> httpclient.maxTotalConnections = 20000
>
> is defaultMaxPerRoute is per third party service we are calling ? if yes
> then
> if there are 4 services we have to call then settings should be
> httpclient.defaultMaxPerRoute = 10000
> httpclient.maxTotalConnections = 40000
> Each route getting 10000 , is that assumption correct ?
>
>
>> 2. We sometimes get following errors, which are not able to figure out if
>> it
>> is the third party server or httpclient issue. These errors show up
>> during
>> peak volume hitting our servers.
>> a. java.net.SocketException: Connection reset
>
> Someone correct me if I'm wrong, but I believe this means the server
> close the connection on you. This brings up another important
> question, are you making HTTP 1.0 or 1.1 requests? Keep-Alive?
>
> we are using 1.1, keep-alive we left default in connection manager , did
> not change anything there.
>
>> b. :java.net.SocketTimeoutException: Read timed out
>
> This is somewhat straightforward, you tried to read, but didn't get data
> in 20s.
>
>> c. org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting
>> for
>> connection
>
> This is also somewhat straightforward, there was, "A timeout while
> waiting for an available connection from a connection manager." (from
> the Javadocs) How many outstanding requests did you have? Could all
> 1000 be dealing with some long request?
>
> I would also see if you can enable logging (this can kill performance)
> and maybe scale-back your numbers a bit and see if you can get it to
> happen again? My guess is that you're simply exhausting resources (#
> connections, etc). How many requests were you making at peak times?
>
> we want to support upto 4M per hour
>
> Hope this helps...
>
> Bill-
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
>
--
View this message in context: http://old.nabble.com/httpclient.defaultMaxPerRoute-and-maxTotalConnections-questions-tp32921044p32924613.html
Sent from the HttpClient-User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: httpclient.defaultMaxPerRoute and maxTotalConnections questions
Posted by William Speirs <ws...@apache.org>.
On Mon, Dec 5, 2011 at 6:07 PM, mpindyala <mo...@rovicorp.com> wrote:
> We have a java web application which makes calls to 2 of our third party
> Services using httpclient.
>
> Following are the settings for
> org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManagerr. the numbers
> are ridiculously high
> httpclient.defaultMaxPerRoute = 10000
> httpclient.maxTotalConnections = 20000
>
> Socket and connection timeout on org.apache.http.params.HttpConnectionParams
> are set as
> httpclient.socketTimeout = 120000ms
> httpclient.connectionTimeout = 20000ms
>
> we have disabled the
> org.apache.http.params.HttpConnectionParams#setStaleCheckingEnabled = false.
> we have a cron process running which checks for idle connections using
> ClientConnectionManager object every 10 seconds and closes expired & idle
> connections.
What do you mean a "cron process"? A thread that checks? Why would you
do this at all? Why wouldn't you let the connection manager handle
this for you?
> We also have retry handler set on
> org.apache.http.impl.client.DefaultHttpClient. It retries once with a 40ms
> delay.
>
>
> So questions are as follows
> 1. Does above setting look ok for you ? if not what are the optimal numbers.
> Lets say there are 4 third party services we have to call , how should we
> set these numbers to support that.
There are no real "optimal numbers" it is more a matter of the service
level agreement of the 3rd party and the expectations of your program.
You have a socket timeout of 2 minutes and a connection timeout of 20
seconds. Those seem high to me, but again, you might be doing
something which necessitates this.
> 2. We sometimes get following errors, which are not able to figure out if it
> is the third party server or httpclient issue. These errors show up during
> peak volume hitting our servers.
> a. java.net.SocketException: Connection reset
Someone correct me if I'm wrong, but I believe this means the server
close the connection on you. This brings up another important
question, are you making HTTP 1.0 or 1.1 requests? Keep-Alive?
> b. :java.net.SocketTimeoutException: Read timed out
This is somewhat straightforward, you tried to read, but didn't get data in 20s.
> c. org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for
> connection
This is also somewhat straightforward, there was, "A timeout while
waiting for an available connection from a connection manager." (from
the Javadocs) How many outstanding requests did you have? Could all
1000 be dealing with some long request?
I would also see if you can enable logging (this can kill performance)
and maybe scale-back your numbers a bit and see if you can get it to
happen again? My guess is that you're simply exhausting resources (#
connections, etc). How many requests were you making at peak times?
Hope this helps...
Bill-
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org