You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by "Jon Moore (JIRA)" <ji...@apache.org> on 2011/07/15 19:58:00 UTC
[jira] [Created] (HTTPCLIENT-1108) different connection reuse
strategy would reduce number of open connections
different connection reuse strategy would reduce number of open connections
---------------------------------------------------------------------------
Key: HTTPCLIENT-1108
URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1108
Project: HttpComponents HttpClient
Issue Type: Improvement
Components: HttpClient
Affects Versions: 4.1.1
Reporter: Jon Moore
Assignee: Jon Moore
Priority: Minor
Fix For: 4.1.2
Currently, the ThreadSafeClientConnManager reuses persistent connections in a round-robin fashion: when the connection is checked back into the pool, it is added to a queue of available connections for later use. This has the perhaps unintended effect that the client may keep open more connections than needed, because the "last used" time keeps getting updated and none of the connections can get reclaimed via closeIdleConnections().
Exchanging the queue (FIFO) for a stack (LIFO) would result in extra connections actually becoming idle for long enough to be reclaimed.
I have a working patch for this; just need to get my act together to get it in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1108) different connection reuse
strategy would reduce number of open connections
Posted by "Jon Moore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067657#comment-13067657 ]
Jon Moore commented on HTTPCLIENT-1108:
---------------------------------------
@James: that looks very interesting, thanks for posting that. I think the FIFO fix that I checked in is an improvement over LIFO, although it optimizes for minimizing open connections rather than for performance (although I think reusing the most-recently-used connection will tend to have you use the one with the largest CWND anyway).
Would you mind opening a new issue with a feature request for supporting configurable connection reuse strategies? I think that's the right long-term strategy, since clients will want to optimize for different things at different times.
> different connection reuse strategy would reduce number of open connections
> ---------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1108
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1108
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient
> Affects Versions: 4.1.1
> Reporter: Jon Moore
> Assignee: Jon Moore
> Priority: Minor
> Fix For: 4.1.2
>
>
> Currently, the ThreadSafeClientConnManager reuses persistent connections in a round-robin fashion: when the connection is checked back into the pool, it is added to a queue of available connections for later use. This has the perhaps unintended effect that the client may keep open more connections than needed, because the "last used" time keeps getting updated and none of the connections can get reclaimed via closeIdleConnections().
> Exchanging the queue (FIFO) for a stack (LIFO) would result in extra connections actually becoming idle for long enough to be reclaimed.
> I have a working patch for this; just need to get my act together to get it in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Updated] (HTTPCLIENT-1108) different connection reuse
strategy would reduce number of open connections
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Oleg Kalnichevski updated HTTPCLIENT-1108:
------------------------------------------
Fix Version/s: (was: 4.1.2)
4.2 Alpha1
> different connection reuse strategy would reduce number of open connections
> ---------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1108
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1108
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient
> Affects Versions: 4.1.1
> Reporter: Jon Moore
> Assignee: Jon Moore
> Priority: Minor
> Fix For: 4.2 Alpha1
>
>
> Currently, the ThreadSafeClientConnManager reuses persistent connections in a round-robin fashion: when the connection is checked back into the pool, it is added to a queue of available connections for later use. This has the perhaps unintended effect that the client may keep open more connections than needed, because the "last used" time keeps getting updated and none of the connections can get reclaimed via closeIdleConnections().
> Exchanging the queue (FIFO) for a stack (LIFO) would result in extra connections actually becoming idle for long enough to be reclaimed.
> I have a working patch for this; just need to get my act together to get it in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-1108) different connection reuse
strategy would reduce number of open connections
Posted by "Jon Moore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jon Moore resolved HTTPCLIENT-1108.
-----------------------------------
Resolution: Invalid
Fix Version/s: (was: 4.2 Alpha1)
Upon further review, the implementation already does the right thing; I was confused by the free connection management between ConnPoolPerRoute and RouteSpecificPool. The RouteSpecificPool *does* manage the free connections in LIFO order already (and there's even a comment that this is to promote unneeded connections becoming idle); on the other hand, the ConnPoolPerRoute uses FIFO when it "steals" a connection from one route-specific pool to give to another, as that's the connection that is least likely to be missed/needed for that route.
> different connection reuse strategy would reduce number of open connections
> ---------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1108
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1108
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient
> Affects Versions: 4.1.1
> Reporter: Jon Moore
> Assignee: Jon Moore
> Priority: Minor
>
> Currently, the ThreadSafeClientConnManager reuses persistent connections in a round-robin fashion: when the connection is checked back into the pool, it is added to a queue of available connections for later use. This has the perhaps unintended effect that the client may keep open more connections than needed, because the "last used" time keeps getting updated and none of the connections can get reclaimed via closeIdleConnections().
> Exchanging the queue (FIFO) for a stack (LIFO) would result in extra connections actually becoming idle for long enough to be reclaimed.
> I have a working patch for this; just need to get my act together to get it in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1108) different connection reuse
strategy would reduce number of open connections
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13066153#comment-13066153 ]
Oleg Kalnichevski commented on HTTPCLIENT-1108:
-----------------------------------------------
The trunk, not the 4.1.x branch, right? Ideally the adaptive connection pool stuff should go through the standard alpha / beta / stable QA process and is targeted for 4.2.
Oleg
> different connection reuse strategy would reduce number of open connections
> ---------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1108
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1108
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient
> Affects Versions: 4.1.1
> Reporter: Jon Moore
> Assignee: Jon Moore
> Priority: Minor
> Fix For: 4.1.2
>
>
> Currently, the ThreadSafeClientConnManager reuses persistent connections in a round-robin fashion: when the connection is checked back into the pool, it is added to a queue of available connections for later use. This has the perhaps unintended effect that the client may keep open more connections than needed, because the "last used" time keeps getting updated and none of the connections can get reclaimed via closeIdleConnections().
> Exchanging the queue (FIFO) for a stack (LIFO) would result in extra connections actually becoming idle for long enough to be reclaimed.
> I have a working patch for this; just need to get my act together to get it in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1108) different connection reuse
strategy would reduce number of open connections
Posted by "James Abley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067335#comment-13067335 ]
James Abley commented on HTTPCLIENT-1108:
-----------------------------------------
One minor observation. From a performance perspective, a different data structure might be preferable. I was thinking of something that sorted the connections based on CWND might be better, similar to what recently landed in Firefox [1],[2]
[1] https://bugzilla.mozilla.org/show_bug.cgi?id=624739
[2] http://blog.httpwatch.com/2011/06/10/investigating-the-network-performance-of-firefox-5/
> different connection reuse strategy would reduce number of open connections
> ---------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1108
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1108
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient
> Affects Versions: 4.1.1
> Reporter: Jon Moore
> Assignee: Jon Moore
> Priority: Minor
> Fix For: 4.1.2
>
>
> Currently, the ThreadSafeClientConnManager reuses persistent connections in a round-robin fashion: when the connection is checked back into the pool, it is added to a queue of available connections for later use. This has the perhaps unintended effect that the client may keep open more connections than needed, because the "last used" time keeps getting updated and none of the connections can get reclaimed via closeIdleConnections().
> Exchanging the queue (FIFO) for a stack (LIFO) would result in extra connections actually becoming idle for long enough to be reclaimed.
> I have a working patch for this; just need to get my act together to get it in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-1108) different connection reuse
strategy would reduce number of open connections
Posted by "Jon Moore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jon Moore resolved HTTPCLIENT-1108.
-----------------------------------
Resolution: Fixed
> different connection reuse strategy would reduce number of open connections
> ---------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1108
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1108
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient
> Affects Versions: 4.1.1
> Reporter: Jon Moore
> Assignee: Jon Moore
> Priority: Minor
> Fix For: 4.1.2
>
>
> Currently, the ThreadSafeClientConnManager reuses persistent connections in a round-robin fashion: when the connection is checked back into the pool, it is added to a queue of available connections for later use. This has the perhaps unintended effect that the client may keep open more connections than needed, because the "last used" time keeps getting updated and none of the connections can get reclaimed via closeIdleConnections().
> Exchanging the queue (FIFO) for a stack (LIFO) would result in extra connections actually becoming idle for long enough to be reclaimed.
> I have a working patch for this; just need to get my act together to get it in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1108) different connection reuse
strategy would reduce number of open connections
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13066140#comment-13066140 ]
Oleg Kalnichevski commented on HTTPCLIENT-1108:
-----------------------------------------------
Great! Could you please merge the fix down to the 4.1.x branch as well?
Oleg
> different connection reuse strategy would reduce number of open connections
> ---------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1108
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1108
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient
> Affects Versions: 4.1.1
> Reporter: Jon Moore
> Assignee: Jon Moore
> Priority: Minor
> Fix For: 4.1.2
>
>
> Currently, the ThreadSafeClientConnManager reuses persistent connections in a round-robin fashion: when the connection is checked back into the pool, it is added to a queue of available connections for later use. This has the perhaps unintended effect that the client may keep open more connections than needed, because the "last used" time keeps getting updated and none of the connections can get reclaimed via closeIdleConnections().
> Exchanging the queue (FIFO) for a stack (LIFO) would result in extra connections actually becoming idle for long enough to be reclaimed.
> I have a working patch for this; just need to get my act together to get it in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Closed] (HTTPCLIENT-1108) different connection reuse
strategy would reduce number of open connections
Posted by "Jon Moore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jon Moore closed HTTPCLIENT-1108.
---------------------------------
> different connection reuse strategy would reduce number of open connections
> ---------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1108
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1108
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient
> Affects Versions: 4.1.1
> Reporter: Jon Moore
> Assignee: Jon Moore
> Priority: Minor
> Fix For: 4.1.2
>
>
> Currently, the ThreadSafeClientConnManager reuses persistent connections in a round-robin fashion: when the connection is checked back into the pool, it is added to a queue of available connections for later use. This has the perhaps unintended effect that the client may keep open more connections than needed, because the "last used" time keeps getting updated and none of the connections can get reclaimed via closeIdleConnections().
> Exchanging the queue (FIFO) for a stack (LIFO) would result in extra connections actually becoming idle for long enough to be reclaimed.
> I have a working patch for this; just need to get my act together to get it in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Reopened] (HTTPCLIENT-1108) different connection reuse
strategy would reduce number of open connections
Posted by "Jon Moore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jon Moore reopened HTTPCLIENT-1108:
-----------------------------------
Reopening issue to incorporate Oleg's feedback.
> different connection reuse strategy would reduce number of open connections
> ---------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1108
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1108
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient
> Affects Versions: 4.1.1
> Reporter: Jon Moore
> Assignee: Jon Moore
> Priority: Minor
> Fix For: 4.1.2
>
>
> Currently, the ThreadSafeClientConnManager reuses persistent connections in a round-robin fashion: when the connection is checked back into the pool, it is added to a queue of available connections for later use. This has the perhaps unintended effect that the client may keep open more connections than needed, because the "last used" time keeps getting updated and none of the connections can get reclaimed via closeIdleConnections().
> Exchanging the queue (FIFO) for a stack (LIFO) would result in extra connections actually becoming idle for long enough to be reclaimed.
> I have a working patch for this; just need to get my act together to get it in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1108) different connection reuse
strategy would reduce number of open connections
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13069528#comment-13069528 ]
Oleg Kalnichevski commented on HTTPCLIENT-1108:
-----------------------------------------------
@Jon, I suggest we delay the resolution of this issue until 4.2-alpha1. I do not think it should block 4.1.2 any longer.
Oleg
> different connection reuse strategy would reduce number of open connections
> ---------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1108
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1108
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient
> Affects Versions: 4.1.1
> Reporter: Jon Moore
> Assignee: Jon Moore
> Priority: Minor
> Fix For: 4.1.2
>
>
> Currently, the ThreadSafeClientConnManager reuses persistent connections in a round-robin fashion: when the connection is checked back into the pool, it is added to a queue of available connections for later use. This has the perhaps unintended effect that the client may keep open more connections than needed, because the "last used" time keeps getting updated and none of the connections can get reclaimed via closeIdleConnections().
> Exchanging the queue (FIFO) for a stack (LIFO) would result in extra connections actually becoming idle for long enough to be reclaimed.
> I have a working patch for this; just need to get my act together to get it in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1108) different connection reuse
strategy would reduce number of open connections
Posted by "Jon Moore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067102#comment-13067102 ]
Jon Moore commented on HTTPCLIENT-1108:
---------------------------------------
Ok, backported down to 4.1.x.
> different connection reuse strategy would reduce number of open connections
> ---------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1108
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1108
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient
> Affects Versions: 4.1.1
> Reporter: Jon Moore
> Assignee: Jon Moore
> Priority: Minor
> Fix For: 4.1.2
>
>
> Currently, the ThreadSafeClientConnManager reuses persistent connections in a round-robin fashion: when the connection is checked back into the pool, it is added to a queue of available connections for later use. This has the perhaps unintended effect that the client may keep open more connections than needed, because the "last used" time keeps getting updated and none of the connections can get reclaimed via closeIdleConnections().
> Exchanging the queue (FIFO) for a stack (LIFO) would result in extra connections actually becoming idle for long enough to be reclaimed.
> I have a working patch for this; just need to get my act together to get it in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1108) different connection reuse
strategy would reduce number of open connections
Posted by "Jon Moore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067696#comment-13067696 ]
Jon Moore commented on HTTPCLIENT-1108:
---------------------------------------
@Oleg: I'm already doing the test cases anyway just to cement my understanding of the code. Michael Feathers would be proud. :)
> different connection reuse strategy would reduce number of open connections
> ---------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1108
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1108
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient
> Affects Versions: 4.1.1
> Reporter: Jon Moore
> Assignee: Jon Moore
> Priority: Minor
> Fix For: 4.1.2
>
>
> Currently, the ThreadSafeClientConnManager reuses persistent connections in a round-robin fashion: when the connection is checked back into the pool, it is added to a queue of available connections for later use. This has the perhaps unintended effect that the client may keep open more connections than needed, because the "last used" time keeps getting updated and none of the connections can get reclaimed via closeIdleConnections().
> Exchanging the queue (FIFO) for a stack (LIFO) would result in extra connections actually becoming idle for long enough to be reclaimed.
> I have a working patch for this; just need to get my act together to get it in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1108) different connection reuse
strategy would reduce number of open connections
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067689#comment-13067689 ]
Oleg Kalnichevski commented on HTTPCLIENT-1108:
-----------------------------------------------
@Jon
More test cases are needed, for sure. I actually managed to get near 100% test coverage for asynchronous connection pool classes in HttpAsyncClient [1] and was going to port those test cases to the synchronous (blocking) implementation at some point. Do not worry about the tests for the time being. Eventually all low level pooling code should get moved to HttpCore.
Oleg
[1] http://hc.apache.org/httpcomponents-asyncclient-dev/httpasyncclient/clover/org/apache/http/impl/nio/pool/pkg-summary.html
> different connection reuse strategy would reduce number of open connections
> ---------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1108
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1108
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient
> Affects Versions: 4.1.1
> Reporter: Jon Moore
> Assignee: Jon Moore
> Priority: Minor
> Fix For: 4.1.2
>
>
> Currently, the ThreadSafeClientConnManager reuses persistent connections in a round-robin fashion: when the connection is checked back into the pool, it is added to a queue of available connections for later use. This has the perhaps unintended effect that the client may keep open more connections than needed, because the "last used" time keeps getting updated and none of the connections can get reclaimed via closeIdleConnections().
> Exchanging the queue (FIFO) for a stack (LIFO) would result in extra connections actually becoming idle for long enough to be reclaimed.
> I have a working patch for this; just need to get my act together to get it in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Closed] (HTTPCLIENT-1108) different connection reuse
strategy would reduce number of open connections
Posted by "Jon Moore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jon Moore closed HTTPCLIENT-1108.
---------------------------------
> different connection reuse strategy would reduce number of open connections
> ---------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1108
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1108
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient
> Affects Versions: 4.1.1
> Reporter: Jon Moore
> Assignee: Jon Moore
> Priority: Minor
>
> Currently, the ThreadSafeClientConnManager reuses persistent connections in a round-robin fashion: when the connection is checked back into the pool, it is added to a queue of available connections for later use. This has the perhaps unintended effect that the client may keep open more connections than needed, because the "last used" time keeps getting updated and none of the connections can get reclaimed via closeIdleConnections().
> Exchanging the queue (FIFO) for a stack (LIFO) would result in extra connections actually becoming idle for long enough to be reclaimed.
> I have a working patch for this; just need to get my act together to get it in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1108) different connection reuse
strategy would reduce number of open connections
Posted by "Jon Moore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13069608#comment-13069608 ]
Jon Moore commented on HTTPCLIENT-1108:
---------------------------------------
@Oleg: I'm fine with delaying this fix till after 4.1.2. I think it's going to take longer than I thought originally.
> different connection reuse strategy would reduce number of open connections
> ---------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1108
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1108
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient
> Affects Versions: 4.1.1
> Reporter: Jon Moore
> Assignee: Jon Moore
> Priority: Minor
> Fix For: 4.1.2
>
>
> Currently, the ThreadSafeClientConnManager reuses persistent connections in a round-robin fashion: when the connection is checked back into the pool, it is added to a queue of available connections for later use. This has the perhaps unintended effect that the client may keep open more connections than needed, because the "last used" time keeps getting updated and none of the connections can get reclaimed via closeIdleConnections().
> Exchanging the queue (FIFO) for a stack (LIFO) would result in extra connections actually becoming idle for long enough to be reclaimed.
> I have a working patch for this; just need to get my act together to get it in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1108) different connection reuse
strategy would reduce number of open connections
Posted by "Jon Moore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067087#comment-13067087 ]
Jon Moore commented on HTTPCLIENT-1108:
---------------------------------------
Agreed, adaptive connpool should go to 4.2 and not 4.1.x.
> different connection reuse strategy would reduce number of open connections
> ---------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1108
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1108
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient
> Affects Versions: 4.1.1
> Reporter: Jon Moore
> Assignee: Jon Moore
> Priority: Minor
> Fix For: 4.1.2
>
>
> Currently, the ThreadSafeClientConnManager reuses persistent connections in a round-robin fashion: when the connection is checked back into the pool, it is added to a queue of available connections for later use. This has the perhaps unintended effect that the client may keep open more connections than needed, because the "last used" time keeps getting updated and none of the connections can get reclaimed via closeIdleConnections().
> Exchanging the queue (FIFO) for a stack (LIFO) would result in extra connections actually becoming idle for long enough to be reclaimed.
> I have a working patch for this; just need to get my act together to get it in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1108) different connection reuse
strategy would reduce number of open connections
Posted by "Jon Moore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067677#comment-13067677 ]
Jon Moore commented on HTTPCLIENT-1108:
---------------------------------------
@Oleg: regarding #deleteLeastUsedEntry, yes, you're right. I think there're some unit tests missing here that I'll try to add while I fix this.
Regarding the data structure, let me get the functionality fixed first and then we can discuss implementation and data structure. I think you've pointed out that whatever we use, we need to be able to remove from both ends of the list, depending on whether we want the most-recently or least-recently used connection.
> different connection reuse strategy would reduce number of open connections
> ---------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1108
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1108
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient
> Affects Versions: 4.1.1
> Reporter: Jon Moore
> Assignee: Jon Moore
> Priority: Minor
> Fix For: 4.1.2
>
>
> Currently, the ThreadSafeClientConnManager reuses persistent connections in a round-robin fashion: when the connection is checked back into the pool, it is added to a queue of available connections for later use. This has the perhaps unintended effect that the client may keep open more connections than needed, because the "last used" time keeps getting updated and none of the connections can get reclaimed via closeIdleConnections().
> Exchanging the queue (FIFO) for a stack (LIFO) would result in extra connections actually becoming idle for long enough to be reclaimed.
> I have a working patch for this; just need to get my act together to get it in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1108) different connection reuse
strategy would reduce number of open connections
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067663#comment-13067663 ]
Oleg Kalnichevski commented on HTTPCLIENT-1108:
-----------------------------------------------
@Jon
I spotted a few problems with the latest changes. Firstly, it appears the behaviour of #deleteLeastUsedEntry no longer corresponds to its contract. Does not it remove the most used entry now instead of the least one? Secondly, Stack is effectively a Vector. Methods of Vector class are synchronised, which is an unnecessary overhead as far as I can tell, since the internal structures of the class are guarded by poolLock. Wouldn't it be more efficient to use LinkedList instead of Vector?
Oleg
> different connection reuse strategy would reduce number of open connections
> ---------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1108
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1108
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient
> Affects Versions: 4.1.1
> Reporter: Jon Moore
> Assignee: Jon Moore
> Priority: Minor
> Fix For: 4.1.2
>
>
> Currently, the ThreadSafeClientConnManager reuses persistent connections in a round-robin fashion: when the connection is checked back into the pool, it is added to a queue of available connections for later use. This has the perhaps unintended effect that the client may keep open more connections than needed, because the "last used" time keeps getting updated and none of the connections can get reclaimed via closeIdleConnections().
> Exchanging the queue (FIFO) for a stack (LIFO) would result in extra connections actually becoming idle for long enough to be reclaimed.
> I have a working patch for this; just need to get my act together to get it in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1108) different connection reuse
strategy would reduce number of open connections
Posted by "Jon Moore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13066150#comment-13066150 ]
Jon Moore commented on HTTPCLIENT-1108:
---------------------------------------
Sure, will do. Am in the middle of trying to merge the adaptive connection pool stuff back down too.
> different connection reuse strategy would reduce number of open connections
> ---------------------------------------------------------------------------
>
> Key: HTTPCLIENT-1108
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1108
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient
> Affects Versions: 4.1.1
> Reporter: Jon Moore
> Assignee: Jon Moore
> Priority: Minor
> Fix For: 4.1.2
>
>
> Currently, the ThreadSafeClientConnManager reuses persistent connections in a round-robin fashion: when the connection is checked back into the pool, it is added to a queue of available connections for later use. This has the perhaps unintended effect that the client may keep open more connections than needed, because the "last used" time keeps getting updated and none of the connections can get reclaimed via closeIdleConnections().
> Exchanging the queue (FIFO) for a stack (LIFO) would result in extra connections actually becoming idle for long enough to be reclaimed.
> I have a working patch for this; just need to get my act together to get it in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org