You are viewing a plain text version of this content. The canonical link for it is here.

Posted to httpclient-users@hc.apache.org by Joan Balagueró <jo...@grupoventus.com> on 2009/02/05 19:28:00 UTC

MAX CONNECTIONS

Hello,

 

Im testing HttpClient4, and Ive found the following problem.

 

I set MaxTotalConnections to Integer.MAX_VALUE:

ConnManagerParams.setMaxTotalConnections(this.objHttpParams,
Integer.MAX_VALUE);

 

But when I send 5 simultaneous requests, I get 2 maxConnections errors.
The exception is:

 

org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for
connection

        at
org.apache.http.impl.conn.tsccm.ConnPoolByRoute.getEntryBlocking(ConnPoolByR
oute.java:353)

        at
org.apache.http.impl.conn.tsccm.ConnPoolByRoute$1.getPoolEntry(ConnPoolByRou
te.java:238)

        at
org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager$1.getConnection(
ThreadSafeClientConnManager.java:175)

        at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDir
ector.java:324)

        at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.ja
va:555)

        at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.ja
va:487)

        at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.ja
va:465)

        at
com.vpfw.proxy.services.http.HttpService.sendRequest(HttpService.java:565)

        at
com.vpfw.proxy.services.http.HttpService.connectServer(HttpService.java:301)

        at
com.vpfw.proxy.services.http.HttpService.connect(HttpService.java:259)

        at com.vpfw.proxy.servlet.ProxyUtils.byPass(ProxyUtils.java:216)

        at
com.vpfw.proxy.servlet.ProxyServlet.service(ProxyServlet.java:449)

        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)

 

 

If I print the maxConnections value inside the exception
(System.out.println(ConnManagerParams.getMaxTotalConnections(this.objHttpPar
ams))), I get: 2147483647. Thats OK.

 

I suppose Im forgetting something. Maybe
ConnManagerParams.setMaxConnectionsPerRoute(). Any help?

 

Thanks,

 

Joan.

Re: MAX CONNECTIONS / reusable connections

Posted by jpswain <jp...@gmail.com>.

Ok, I'm back to answer my own question :)
I figured out that this will let me increase the number of simultaneous
connections per route and total:

        // >> ConnectionManager
        // Setting max connections per route.
        ConnPerRoute connPerRoute = new ConnPerRouteBean(12);
        ConnManagerParams.setMaxConnectionsPerRoute(params, connPerRoute);
        ConnManagerParams.setMaxTotalConnections(params, 20);
        
        connMan = new ThreadSafeClientConnManager(params, schemeRegistry);

Also, it seems like the Twitter API simply doesn't allow keep alive.



jpswain wrote:
> 
> Hi, 
> Oleg, I think your answer explains why I can't seem to get more than 2
> simultaneous connections during my development here.  I can't seem to find
> out how to se the max connections per route.  Could you tell me how to do
> this?
> 
> Also, I'm running logging in debug mode so I can see all the "wire" and
> header statements, and even though all my requests to the Twitter REST API
> say "Keep-Alive" and I get "Keep-Alive" responses in the headers I can't
> seem to find a way to reuse any connections.  
> 
> 

-- 
View this message in context: http://www.nabble.com/MAX-CONNECTIONS-tp21858014p22964871.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org

Re: MAX CONNECTIONS / reusable connections

Posted by jpswain <jp...@gmail.com>.

Hi, 
Oleg, I think your answer explains why I can't seem to get more than 2
simultaneous connections during my development here.  I can't seem to find
out how to se the max connections per route.  Could you tell me how to do
this?

Also, I'm running logging in debug mode so I can see all the "wire" and
header statements, and even though all my requests to the Twitter REST API
say "Keep-Alive" and I get "Keep-Alive" responses in the headers I can't
seem to find a way to reuse any connections.  

Does this line taken from below indicated that the their server simply
doesn't support letting me reuse connections?
18:00:02,490 DEBUG ThreadSafeClientConnManager:223 - Released connection is
not reusable.




Here are some of the lines from the logger that might be relevant:
17:59:59,364 DEBUG ThreadSafeClientConnManager:171 -
ThreadSafeClientConnManager.getConnection:
HttpRoute[{s}->https://twitter.com], timeout = 0
17:59:59,366 DEBUG ConnPoolByRoute:289 - Total connections kept alive: 0
17:59:59,366 DEBUG ConnPoolByRoute:290 - Total issued connections: 0
17:59:59,367 DEBUG ConnPoolByRoute:291 - Total allocated connection: 0 out
of 20
17:59:59,368 DEBUG ConnPoolByRoute:462 - No free connections
[HttpRoute[{s}->https://twitter.com]][null]
17:59:59,368 DEBUG ConnPoolByRoute:308 - Available capacity: 2 out of 2
[HttpRoute[{s}->https://twitter.com]][null]
17:59:59,368 DEBUG ConnPoolByRoute:489 - Creating new connection
[HttpRoute[{s}->https://twitter.com]]

...

18:00:00,216 DEBUG DefaultRequestDirector:408 - Attempt 1 to execute request
18:00:00,246 DEBUG headers:251 - >> GET /account/verify_credentials.xml
HTTP/1.1
18:00:00,246 DEBUG headers:254 - >> Authorization: Basic [ cut out ]
18:00:00,246 DEBUG headers:254 - >> Host: twitter.com
18:00:00,247 DEBUG headers:254 - >> Connection: Keep-Alive

...

18:00:02,429 DEBUG headers:237 - << HTTP/1.1 200 OK
18:00:02,429 DEBUG headers:240 - << Date: Wed, 08 Apr 2009 23:00:02 GMT
18:00:02,429 DEBUG headers:240 - << Server: hi
18:00:02,429 DEBUG headers:240 - << Last-Modified: Wed, 08 Apr 2009 23:00:03
GMT
18:00:02,430 DEBUG headers:240 - << Status: 200 OK
18:00:02,430 DEBUG headers:240 - << ETag: "1fe1e2a6753e8ae25c3e889a8b31b160"
18:00:02,430 DEBUG headers:240 - << Pragma: no-cache
18:00:02,430 DEBUG headers:240 - << Cache-Control: no-cache, no-store,
must-revalidate, pre-check=0, post-check=0
18:00:02,431 DEBUG headers:240 - << Content-Type: application/xml;
charset=utf-8
18:00:02,431 DEBUG headers:240 - << Content-Length: 1621
18:00:02,431 DEBUG headers:240 - << Expires: Tue, 31 Mar 1981 05:00:00 GMT
18:00:02,431 DEBUG headers:240 - << X-Revision:
b1c9a7d2d8022081fa8a4fffe69991d16fa0f5df
18:00:02,431 DEBUG headers:240 - << X-Transaction: 1239231603-91693-5485
18:00:02,438 DEBUG headers:240 - << Set-Cookie: lang=en; path=/
18:00:02,439 DEBUG headers:240 - << Set-Cookie: lang=en; path=/
18:00:02,439 DEBUG headers:240 - << Set-Cookie: _twitter_sess=[ cut out ];
domain=.twitter.com; path=/
18:00:02,441 DEBUG headers:240 - << Vary: Accept-Encoding
18:00:02,443 DEBUG headers:240 - << Connection: close

...

18:00:02,486 DEBUG DefaultClientConnection:147 - Connection shut down
18:00:02,490 DEBUG ThreadSafeClientConnManager:223 - Released connection is
not reusable.
18:00:02,491 DEBUG ConnPoolByRoute:374 - Releasing connection
[HttpRoute[{s}->https://twitter.com]][null]
18:00:02,491 DEBUG ConnPoolByRoute:631 - Notifying no-one, there are no
waiting threads




olegk wrote:
> 
> 
> That's exactly the case. Per requirements of the HTTP spec HTTP agents 
> may not be using more than 2 concurrent connections to the same host. To 
> override this behavior set the max connection per route parameter to a 
> greater number.
> 
> Hope this helps
> 
> Oleg
> 
> 

-- 
View this message in context: http://www.nabble.com/MAX-CONNECTIONS-tp21858014p22961572.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org

Re: MAX CONNECTIONS

Posted by Oleg Kalnichevski <ol...@apache.org>.

Joan Balagueró wrote:
> Hello,
> 
>  
> 
> I’m testing HttpClient4, and I’ve found the following problem.
> 
>  
> 
> I set MaxTotalConnections to Integer.MAX_VALUE:
> 
> ConnManagerParams.setMaxTotalConnections(this.objHttpParams,
> Integer.MAX_VALUE);
> 
>  
> 
> But when I send 5 simultaneous requests, I get 2 “maxConnections” errors.
> The exception is:
> 

...

> 
> I suppose I’m forgetting something. Maybe
> “ConnManagerParams.setMaxConnectionsPerRoute()”. Any help?
> 
>  

That's exactly the case. Per requirements of the HTTP spec HTTP agents 
may not be using more than 2 concurrent connections to the same host. To 
override this behavior set the max connection per route parameter to a 
greater number.

Hope this helps

Oleg


> 
> Thanks,
> 
>  
> 
> Joan.
> 
>  
> 
>  
> 
>  
> 
>  
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org