You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by "Anthony Bigbee (JIRA)" <ji...@apache.org> on 2007/01/17 23:23:29 UTC
[jira] Created: (HTTPCLIENT-623)
MultiThreadedHttpConnectionManager exceeds MaxTotalConnections
MultiThreadedHttpConnectionManager exceeds MaxTotalConnections
---------------------------------------------------------------
Key: HTTPCLIENT-623
URL: https://issues.apache.org/jira/browse/HTTPCLIENT-623
Project: HttpComponents HttpClient
Issue Type: Bug
Components: HttpClient
Affects Versions: 3.1 Beta 1
Environment: Solaris 10 SPARC, JDK 1.5.0_10
Reporter: Anthony Bigbee
Priority: Critical
Multithreaded app that uses MultiThreadedHttpConnectionManager and GetMethod is exceeding MaxTotalConnections specified in MultiThreadedHttpConnectionManager.
Connections may be unncessarily created instead of reused, but I do not know if this is the case or whether the MaxTotalConnections is not being honored as the cause of this behavior.
netstat reveals that the number of http TIME_WAIT state sockets exceeds MaxTotalConnections after a short time period for a small number of threads / GetMethod objects. The threads with the GetMethod objects *are* invoking GetMethod.releaseConnection().
This behavior is reproducible every time.
Relevant code snippets below:
httpConnectionManagerParams.setMaxTotalConnections(MAXCONNECTIONS); //MAXCONNECTIONS = 50 for testing
this.connectionManager = new MultiThreadedHttpConnectionManager();
this.connectionManager.setParams(httpConnectionManagerParams);
this.httpClient = new HttpClient(this.connectionManager);
this.httpClient.getParams().setParameter("http.useragent", "Internet Explorer 6.x");
this.httpClient.getParams().setParameter("http.protocol.version", HttpVersion.HTTP_1_1);
this.httpClient.getParams().setParameter("http.connection-manager.max-total", MAXCONNECTIONS);
this.httpClient.getParams().setParameter("http.connection.stalecheck", true);
this.httpClient.getParams().setParameter("http.connection-manager.max-per-host", MAXCONNECTIONSPERHOST);
This bug is reproducible every time.
AJB
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
[jira] Resolved: (HTTPCLIENT-623)
MultiThreadedHttpConnectionManager exceeds MaxTotalConnections
Posted by "Ortwin Glück (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ortwin Glück resolved HTTPCLIENT-623.
-------------------------------------
Resolution: Invalid
Sorry, we are out of luck here.
See API Doc in java.net.SocketOptions:
"Sets SO_REUSEADDR for a socket. This is used only for MulticastSockets in java, and it is set by default for MulticastSockets."
> MultiThreadedHttpConnectionManager exceeds MaxTotalConnections
> ---------------------------------------------------------------
>
> Key: HTTPCLIENT-623
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-623
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1 Beta 1
> Environment: Solaris 10 SPARC, JDK 1.5.0_10
> Reporter: Anthony Bigbee
> Assigned To: Ortwin Glück
> Priority: Critical
>
> Multithreaded app that uses MultiThreadedHttpConnectionManager and GetMethod is exceeding MaxTotalConnections specified in MultiThreadedHttpConnectionManager.
> Connections may be unncessarily created instead of reused, but I do not know if this is the case or whether the MaxTotalConnections is not being honored as the cause of this behavior.
> netstat reveals that the number of http TIME_WAIT state sockets exceeds MaxTotalConnections after a short time period for a small number of threads / GetMethod objects. The threads with the GetMethod objects *are* invoking GetMethod.releaseConnection().
> This behavior is reproducible every time.
> Relevant code snippets below:
> httpConnectionManagerParams.setMaxTotalConnections(MAXCONNECTIONS); //MAXCONNECTIONS = 50 for testing
>
> this.connectionManager = new MultiThreadedHttpConnectionManager();
> this.connectionManager.setParams(httpConnectionManagerParams);
> this.httpClient = new HttpClient(this.connectionManager);
> this.httpClient.getParams().setParameter("http.useragent", "Internet Explorer 6.x");
> this.httpClient.getParams().setParameter("http.protocol.version", HttpVersion.HTTP_1_1);
> this.httpClient.getParams().setParameter("http.connection-manager.max-total", MAXCONNECTIONS);
> this.httpClient.getParams().setParameter("http.connection.stalecheck", true);
> this.httpClient.getParams().setParameter("http.connection-manager.max-per-host", MAXCONNECTIONSPERHOST);
> This bug is reproducible every time.
> AJB
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
[jira] Commented: (HTTPCLIENT-623)
MultiThreadedHttpConnectionManager exceeds MaxTotalConnections
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12466010 ]
Oleg Kalnichevski commented on HTTPCLIENT-623:
----------------------------------------------
> I request a programmatic way to do it via the connection parameters
Anthony,
Consider implementing a custom ProtocolSocketFactory and initialize the socket as you see fit
Oleg
> MultiThreadedHttpConnectionManager exceeds MaxTotalConnections
> ---------------------------------------------------------------
>
> Key: HTTPCLIENT-623
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-623
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1 Beta 1
> Environment: Solaris 10 SPARC, JDK 1.5.0_10
> Reporter: Anthony Bigbee
> Assigned To: Ortwin Glück
> Priority: Critical
>
> Multithreaded app that uses MultiThreadedHttpConnectionManager and GetMethod is exceeding MaxTotalConnections specified in MultiThreadedHttpConnectionManager.
> Connections may be unncessarily created instead of reused, but I do not know if this is the case or whether the MaxTotalConnections is not being honored as the cause of this behavior.
> netstat reveals that the number of http TIME_WAIT state sockets exceeds MaxTotalConnections after a short time period for a small number of threads / GetMethod objects. The threads with the GetMethod objects *are* invoking GetMethod.releaseConnection().
> This behavior is reproducible every time.
> Relevant code snippets below:
> httpConnectionManagerParams.setMaxTotalConnections(MAXCONNECTIONS); //MAXCONNECTIONS = 50 for testing
>
> this.connectionManager = new MultiThreadedHttpConnectionManager();
> this.connectionManager.setParams(httpConnectionManagerParams);
> this.httpClient = new HttpClient(this.connectionManager);
> this.httpClient.getParams().setParameter("http.useragent", "Internet Explorer 6.x");
> this.httpClient.getParams().setParameter("http.protocol.version", HttpVersion.HTTP_1_1);
> this.httpClient.getParams().setParameter("http.connection-manager.max-total", MAXCONNECTIONS);
> this.httpClient.getParams().setParameter("http.connection.stalecheck", true);
> this.httpClient.getParams().setParameter("http.connection-manager.max-per-host", MAXCONNECTIONSPERHOST);
> This bug is reproducible every time.
> AJB
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
[jira] Resolved: (HTTPCLIENT-623)
MultiThreadedHttpConnectionManager exceeds MaxTotalConnections
Posted by "Ortwin Glück (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ortwin Glück resolved HTTPCLIENT-623.
-------------------------------------
Resolution: Invalid
Sockets in TIME_WAIT have been closed. Look it up on Google, please. You must count only the ESTABLISHED connections.
> MultiThreadedHttpConnectionManager exceeds MaxTotalConnections
> ---------------------------------------------------------------
>
> Key: HTTPCLIENT-623
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-623
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1 Beta 1
> Environment: Solaris 10 SPARC, JDK 1.5.0_10
> Reporter: Anthony Bigbee
> Priority: Critical
>
> Multithreaded app that uses MultiThreadedHttpConnectionManager and GetMethod is exceeding MaxTotalConnections specified in MultiThreadedHttpConnectionManager.
> Connections may be unncessarily created instead of reused, but I do not know if this is the case or whether the MaxTotalConnections is not being honored as the cause of this behavior.
> netstat reveals that the number of http TIME_WAIT state sockets exceeds MaxTotalConnections after a short time period for a small number of threads / GetMethod objects. The threads with the GetMethod objects *are* invoking GetMethod.releaseConnection().
> This behavior is reproducible every time.
> Relevant code snippets below:
> httpConnectionManagerParams.setMaxTotalConnections(MAXCONNECTIONS); //MAXCONNECTIONS = 50 for testing
>
> this.connectionManager = new MultiThreadedHttpConnectionManager();
> this.connectionManager.setParams(httpConnectionManagerParams);
> this.httpClient = new HttpClient(this.connectionManager);
> this.httpClient.getParams().setParameter("http.useragent", "Internet Explorer 6.x");
> this.httpClient.getParams().setParameter("http.protocol.version", HttpVersion.HTTP_1_1);
> this.httpClient.getParams().setParameter("http.connection-manager.max-total", MAXCONNECTIONS);
> this.httpClient.getParams().setParameter("http.connection.stalecheck", true);
> this.httpClient.getParams().setParameter("http.connection-manager.max-per-host", MAXCONNECTIONSPERHOST);
> This bug is reproducible every time.
> AJB
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
[jira] Reopened: (HTTPCLIENT-623)
MultiThreadedHttpConnectionManager exceeds MaxTotalConnections
Posted by "Ortwin Glück (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ortwin Glück reopened HTTPCLIENT-623:
-------------------------------------
Assignee: Ortwin Glück
I guess kernel developers will disagree about the necessity of TIME_WAIT *g*. The actual time the kernel waits may be configured (default in Solaris seems to be 4 minutes): http://www.sean.de/Solaris/soltune.html#tcp_time_wait_interval
Socket.setReuseAddress is a JDK 1.4 feature, while HttpClient code is 1.3 compatible. We could use reflection however. I am trying to come up with a patch. In the mean time you could write your own ProtocolSocketFactory implementation that sets all kind of fancy flags on the socket.
> MultiThreadedHttpConnectionManager exceeds MaxTotalConnections
> ---------------------------------------------------------------
>
> Key: HTTPCLIENT-623
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-623
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1 Beta 1
> Environment: Solaris 10 SPARC, JDK 1.5.0_10
> Reporter: Anthony Bigbee
> Assigned To: Ortwin Glück
> Priority: Critical
>
> Multithreaded app that uses MultiThreadedHttpConnectionManager and GetMethod is exceeding MaxTotalConnections specified in MultiThreadedHttpConnectionManager.
> Connections may be unncessarily created instead of reused, but I do not know if this is the case or whether the MaxTotalConnections is not being honored as the cause of this behavior.
> netstat reveals that the number of http TIME_WAIT state sockets exceeds MaxTotalConnections after a short time period for a small number of threads / GetMethod objects. The threads with the GetMethod objects *are* invoking GetMethod.releaseConnection().
> This behavior is reproducible every time.
> Relevant code snippets below:
> httpConnectionManagerParams.setMaxTotalConnections(MAXCONNECTIONS); //MAXCONNECTIONS = 50 for testing
>
> this.connectionManager = new MultiThreadedHttpConnectionManager();
> this.connectionManager.setParams(httpConnectionManagerParams);
> this.httpClient = new HttpClient(this.connectionManager);
> this.httpClient.getParams().setParameter("http.useragent", "Internet Explorer 6.x");
> this.httpClient.getParams().setParameter("http.protocol.version", HttpVersion.HTTP_1_1);
> this.httpClient.getParams().setParameter("http.connection-manager.max-total", MAXCONNECTIONS);
> this.httpClient.getParams().setParameter("http.connection.stalecheck", true);
> this.httpClient.getParams().setParameter("http.connection-manager.max-per-host", MAXCONNECTIONSPERHOST);
> This bug is reproducible every time.
> AJB
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
[jira] Commented: (HTTPCLIENT-623)
MultiThreadedHttpConnectionManager exceeds MaxTotalConnections
Posted by "Anthony Bigbee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12465961 ]
Anthony Bigbee commented on HTTPCLIENT-623:
-------------------------------------------
TIME_WAIT state sockets exceeding MaxTotalConnections are an unnecessary use of system resources but I think can be avoided by calling Socket.setReuseAddress(true) immediately after socket creation (but before binding). I request a programmatic way to do it via the connection parameters and consideration of doing this action by default.
> MultiThreadedHttpConnectionManager exceeds MaxTotalConnections
> ---------------------------------------------------------------
>
> Key: HTTPCLIENT-623
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-623
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1 Beta 1
> Environment: Solaris 10 SPARC, JDK 1.5.0_10
> Reporter: Anthony Bigbee
> Priority: Critical
>
> Multithreaded app that uses MultiThreadedHttpConnectionManager and GetMethod is exceeding MaxTotalConnections specified in MultiThreadedHttpConnectionManager.
> Connections may be unncessarily created instead of reused, but I do not know if this is the case or whether the MaxTotalConnections is not being honored as the cause of this behavior.
> netstat reveals that the number of http TIME_WAIT state sockets exceeds MaxTotalConnections after a short time period for a small number of threads / GetMethod objects. The threads with the GetMethod objects *are* invoking GetMethod.releaseConnection().
> This behavior is reproducible every time.
> Relevant code snippets below:
> httpConnectionManagerParams.setMaxTotalConnections(MAXCONNECTIONS); //MAXCONNECTIONS = 50 for testing
>
> this.connectionManager = new MultiThreadedHttpConnectionManager();
> this.connectionManager.setParams(httpConnectionManagerParams);
> this.httpClient = new HttpClient(this.connectionManager);
> this.httpClient.getParams().setParameter("http.useragent", "Internet Explorer 6.x");
> this.httpClient.getParams().setParameter("http.protocol.version", HttpVersion.HTTP_1_1);
> this.httpClient.getParams().setParameter("http.connection-manager.max-total", MAXCONNECTIONS);
> this.httpClient.getParams().setParameter("http.connection.stalecheck", true);
> this.httpClient.getParams().setParameter("http.connection-manager.max-per-host", MAXCONNECTIONSPERHOST);
> This bug is reproducible every time.
> AJB
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org