You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Christiaan Lamprecht <ch...@googlemail.com> on 2008/01/30 19:10:43 UTC
MTHCM Connection Pool re-use
Hi all,
The question is on how to permanently close a connection in the MTHCM
connection pool.
I'm using the MultiThreadedHttpConnectionManager and each spawned
thread makes a couple of "httpClient.executeMethod(method);" requests.
Now the connection pool never seems to grow beyond a certain point as
each new thread re-uses one of the old connections, if available.
I assume this is because somewhere, apparently through
method.releaseConnection(), MTHCM.releaseConnection(HttpConnection
conn) is called which "Make the given HttpConnection available for use
by other requests. If another thread is blocked in getConnection()
that could use this connection, it will be woken up"
I do also release the connection upon thread completion (Finally ->
method.releaseConnection()) which states that "If the connection can
be reused by other HTTP methods it is NOT closed at this point." i.e
the MTHCM decides.
Re-using a connection means re-using the SSL session associated with
it. I need to create a new session for each client (i.e for each
thread)
Potential solutions:
* Somehow get the HttpConnection associated with the thread and
explicitly close it. (Objects at my disposal: HttpClient, GetMethod
and MTHCM)
* Extend the MTHCM and overwrite the releaseConnection() method. (but
might need access to some private variables and functions)
* Rewrite and compile the MTHCM sourcode. Possibly overwriting
freeConnection (...) in the private class ConnectionPool.
I hope there's an easy solution!
Many thanks again
Christiaan
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: MTHCM Connection Pool re-use
Posted by Christiaan Lamprecht <ch...@googlemail.com>.
Just in case anyone stumbles on the same problem:
Solution is here:
http://www.mail-archive.com/httpclient-user@jakarta.apache.org/msg01028.html
Also;
> hostConfig.setHost("https://mega01.ac.uk/index.html", 8085, new
> Protocol("https", (ProtocolSocketFactory)new
> EasySSLProtocolSocketFactory(), 8085));
>
> in the threads instead, and execute the method on that host instead, I
> get the following error:
also remove https before mega01:
hostConfig.setHost("https://mega01.ac.uk/index.html", .....);
In regard to the first question I did the following (there can of
course be a better solution...)
MTHCM does a very good job at reusing connections for a particular
host configuration, and therefore for a particular PC. The main
concept was to extend HostConfiguration with a unique id to allow
multiple HostConfigurations per target machine, which in turn meant
there could be a unique pool and SSL session for each
HostConfiguration. Also made a few++ other changes.
Thanks again for all the help
Christiaan
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: MTHCM Connection Pool re-use
Posted by Christiaan Lamprecht <ch...@googlemail.com>.
Thanks,
Seems like my assertion;
> > Re-using a connection means re-using the SSL session associated with
> > it. I need to create a new session for each client (i.e for each
> > thread)
.. was only partly true at best. It seems that a new SSL session is
created for every instance of the ProtocolSocketFactory.
So if in the main thread I do;
Protocol.registerProtocol("https", new Protocol("https",
(ProtocolSocketFactory)new EasySSLProtocolSocketFactory(), 8085));
and in each of the spawned threads; (using the MTHCM of course)
hostConfig.setHost("https://mega01.ac.uk/index.html", 8085, "https");
everything works fine. But I would like a new instance of the Protocol
per thread in order to have multiple SSL sessions. So if I remove
Protocol.registerProtocol(...) from the main thread and do;
hostConfig.setHost("https://mega01.ac.uk/index.html", 8085, new
Protocol("https", (ProtocolSocketFactory)new
EasySSLProtocolSocketFactory(), 8085));
in the threads instead, and execute the method on that host instead, I
get the following error:
Fatal transport error: sun.security.validator.ValidatorException: PKIX
path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to
find valid certification path to requested target
javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to
find valid certification path to requested target
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:150)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1518)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:174)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:168)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:848)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:106)
at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:495)
at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:433)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:818)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1030)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:622)
at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:59)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at org.apache.commons.httpclient.HttpConnection.flushRequestOutputStream(HttpConnection.java:828)
at HttpConnectionAdapter.flushRequestOutputStream(HttpConnectionAdapter.java:433)
at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:2116)
at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1096)
at org.apache.commons.httpclient.FluxHttpMethodDirector.executeWithRetry(FluxHttpMethodDirector.java:402)
at org.apache.commons.httpclient.FluxHttpMethodDirector.executeMethod(FluxHttpMethodDirector.java:174)
at SessionThread.executeMethod(SessionThread.java:250)
at SessionThread.run(SessionThread.java:116)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
Caused by: sun.security.validator.ValidatorException: PKIX path
building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to
find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:221)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:145)
at sun.security.validator.Validator.validate(Validator.java:203)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:172)
at com.sun.net.ssl.internal.ssl.JsseX509TrustManager.checkServerTrusted(SSLContextImpl.java:320)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:841)
... 20 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException:
unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:236)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:194)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:216)
... 25 more
Since it works in the first case and not the second I assume I forgot
to initialise something somewhere?
Thanks again
Christiaan
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: MTHCM Connection Pool re-use
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, 2008-01-30 at 18:10 +0000, Christiaan Lamprecht wrote:
> Hi all,
>
> The question is on how to permanently close a connection in the MTHCM
> connection pool.
>
> I'm using the MultiThreadedHttpConnectionManager and each spawned
> thread makes a couple of "httpClient.executeMethod(method);" requests.
>
> Now the connection pool never seems to grow beyond a certain point as
> each new thread re-uses one of the old connections, if available.
>
> I assume this is because somewhere, apparently through
> method.releaseConnection(), MTHCM.releaseConnection(HttpConnection
> conn) is called which "Make the given HttpConnection available for use
> by other requests. If another thread is blocked in getConnection()
> that could use this connection, it will be woken up"
>
> I do also release the connection upon thread completion (Finally ->
> method.releaseConnection()) which states that "If the connection can
> be reused by other HTTP methods it is NOT closed at this point." i.e
> the MTHCM decides.
>
> Re-using a connection means re-using the SSL session associated with
> it. I need to create a new session for each client (i.e for each
> thread)
>
> Potential solutions:
> * Somehow get the HttpConnection associated with the thread and
> explicitly close it. (Objects at my disposal: HttpClient, GetMethod
> and MTHCM)
> * Extend the MTHCM and overwrite the releaseConnection() method. (but
> might need access to some private variables and functions)
Christiaan,
If you just want to make sure that connections get closed upon release,
subclassing MTHCM and overriding releaseConnection() method should be
sufficient.
Oleg
> * Rewrite and compile the MTHCM sourcode. Possibly overwriting
> freeConnection (...) in the private class ConnectionPool.
>
> I hope there's an easy solution!
>
>
> Many thanks again
> Christiaan
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org