You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by "Brooks, Kenneth S" <ke...@chase.com> on 2010/04/21 22:32:23 UTC
Thread to inspect active connections
We are facing an issue where a connection to a server is made, the request is sent, the server sends the response and the client never sees the request.
Ultimately we get a socket timeout because httpclient eventually gives up on that request.
The big problem we have is that this only happens about once every couple of million requests over a 12 hour period (and not consistently).
What we would like to do is increase the connection timeout to something ridiculously long and put some sort of a watcher thread on the client side that can sort of do what the IdleConnectionTimeoutThread is doing.
But instead of just calling the connectionManager.closeIdleConnections(connectionTimeout); we would like to interrogate each open connection to see how long it has been active.
If it has been active for longer than our idleconnection timeout (currently 20 seconds) then we can perform some action..
So lets say that if it was active for 1 minute then we would be able to know the issue is happening and do something (like take thread dumps).
Without doing that, we only know about the problem *after* it has happened and the socket timeout has already been thrown..
That does me no good.. I want to see inside of the VM *as* its happening.
Any thoughts on how to do this?
-k
This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law. If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED. Although this transmission and
any attachments are believed to be free of any virus or other
defect that might affect any computer system into which it is
received and opened, it is the responsibility of the recipient to
ensure that it is virus free and no responsibility is accepted by
JPMorgan Chase & Co., its subsidiaries and affiliates, as
applicable, for any loss or damage arising in any way from its use.
If you received this transmission in error, please immediately
contact the sender and destroy the material in its entirety,
whether in electronic or hard copy format. Thank you.
RE: Thread to inspect active connections
Posted by "Brooks, Kenneth S" <ke...@chase.com>.
Understood..
Thanks for the quick reply.
I'll first look at trying to see if we're clear to get to 4.x.
If not, then I'll extend 3.1 just for our testing..
-k
-----Original Message-----
From: Oleg Kalnichevski [mailto:olegk@apache.org]
Sent: Thursday, April 22, 2010 9:51 AM
To: HttpClient User Discussion
Subject: RE: Thread to inspect active connections
On Thu, 2010-04-22 at 09:29 -0400, Brooks, Kenneth S wrote:
> Sorry.. that was pretty silly not to include the version wasn't it?
>
> Currently 3.1.
>
> Some History:
> We are doing serialization over http.. and thus we are giving out client libraries.
> This means that we had clients that were using jdk1.4..
> That prevented us from using 4.x because of the jdk5 dependency.
> That situation may have changed because I think *almost* everyone is now finally on jdk5..
> I'll look at the feasibility of moving to 4.x
>
> In case I can't I'll still ask, is it possible to do what I am asking in 3.1 today?
>
I am afraid only by forking HttpClient. The 3.x API is simply not
flexible enough.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law. If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED. Although this transmission and
any attachments are believed to be free of any virus or other
defect that might affect any computer system into which it is
received and opened, it is the responsibility of the recipient to
ensure that it is virus free and no responsibility is accepted by
JPMorgan Chase & Co., its subsidiaries and affiliates, as
applicable, for any loss or damage arising in any way from its use.
If you received this transmission in error, please immediately
contact the sender and destroy the material in its entirety,
whether in electronic or hard copy format. Thank you.
RE: Thread to inspect active connections
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Thu, 2010-04-22 at 09:29 -0400, Brooks, Kenneth S wrote:
> Sorry.. that was pretty silly not to include the version wasn't it?
>
> Currently 3.1.
>
> Some History:
> We are doing serialization over http.. and thus we are giving out client libraries.
> This means that we had clients that were using jdk1.4..
> That prevented us from using 4.x because of the jdk5 dependency.
> That situation may have changed because I think *almost* everyone is now finally on jdk5..
> I'll look at the feasibility of moving to 4.x
>
> In case I can't I'll still ask, is it possible to do what I am asking in 3.1 today?
>
I am afraid only by forking HttpClient. The 3.x API is simply not
flexible enough.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: Thread to inspect active connections
Posted by "Brooks, Kenneth S" <ke...@chase.com>.
Sorry.. that was pretty silly not to include the version wasn't it?
Currently 3.1.
Some History:
We are doing serialization over http.. and thus we are giving out client libraries.
This means that we had clients that were using jdk1.4..
That prevented us from using 4.x because of the jdk5 dependency.
That situation may have changed because I think *almost* everyone is now finally on jdk5..
I'll look at the feasibility of moving to 4.x
In case I can't I'll still ask, is it possible to do what I am asking in 3.1 today?
-k
-----Original Message-----
From: Oleg Kalnichevski [mailto:olegk@apache.org]
Sent: Wednesday, April 21, 2010 5:43 PM
To: HttpClient User Discussion
Subject: Re: Thread to inspect active connections
On Wed, 2010-04-21 at 16:32 -0400, Brooks, Kenneth S wrote:
> We are facing an issue where a connection to a server is made, the request is sent, the server sends the response and the client never sees the request.
> Ultimately we get a socket timeout because httpclient eventually gives up on that request.
>
> The big problem we have is that this only happens about once every couple of million requests over a 12 hour period (and not consistently).
>
> What we would like to do is increase the connection timeout to something ridiculously long and put some sort of a watcher thread on the client side that can sort of do what the IdleConnectionTimeoutThread is doing.
> But instead of just calling the connectionManager.closeIdleConnections(connectionTimeout); we would like to interrogate each open connection to see how long it has been active.
> If it has been active for longer than our idleconnection timeout (currently 20 seconds) then we can perform some action..
> So lets say that if it was active for 1 minute then we would be able to know the issue is happening and do something (like take thread dumps).
>
> Without doing that, we only know about the problem *after* it has happened and the socket timeout has already been thrown..
>
> That does me no good.. I want to see inside of the VM *as* its happening.
>
> Any thoughts on how to do this?
>
Kenneth,
What version of HttpClient are you using?
This is certainly doable (especially with HttpClient 4.0) but most
likely would require a custom connection manager. JMX enabled version of
the pooling connection manager would probably be the most sensible
thing.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Thread to inspect active connections
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, 2010-04-21 at 16:32 -0400, Brooks, Kenneth S wrote:
> We are facing an issue where a connection to a server is made, the request is sent, the server sends the response and the client never sees the request.
> Ultimately we get a socket timeout because httpclient eventually gives up on that request.
>
> The big problem we have is that this only happens about once every couple of million requests over a 12 hour period (and not consistently).
>
> What we would like to do is increase the connection timeout to something ridiculously long and put some sort of a watcher thread on the client side that can sort of do what the IdleConnectionTimeoutThread is doing.
> But instead of just calling the connectionManager.closeIdleConnections(connectionTimeout); we would like to interrogate each open connection to see how long it has been active.
> If it has been active for longer than our idleconnection timeout (currently 20 seconds) then we can perform some action..
> So lets say that if it was active for 1 minute then we would be able to know the issue is happening and do something (like take thread dumps).
>
> Without doing that, we only know about the problem *after* it has happened and the socket timeout has already been thrown..
>
> That does me no good.. I want to see inside of the VM *as* its happening.
>
> Any thoughts on how to do this?
>
Kenneth,
What version of HttpClient are you using?
This is certainly doable (especially with HttpClient 4.0) but most
likely would require a custom connection manager. JMX enabled version of
the pooling connection manager would probably be the most sensible
thing.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org