You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by Pankaj Arora <pa...@castiron.com> on 2007/11/12 22:43:36 UTC
Close_Wait problem
Hi,
I am using Http Commons to implement my http client and observe that
after running the client for large number of request connections are
left in close_wait stage. I searched through archives and noticed that
following were suggested:
1. Reuse instances of HttpClient or the associated
HttpConnectionManager, this may have threading implications.
I do that.
2. Do closeidleConnections(long timeout) or customIdleConnectionTimeout
thread
I do closeIdleConnections(20*1000). Please do tell if anything else
needs to be done for this solution.
3. Add a "Connection: close" and/or "Proxy-Connection: close" header to
all methods. This should work in most cases, assuming that the server
echoes the header in the response.
I don't want to do this as can't take the performance hit.
Is there any other way or do I need to do anything else in first 2
solutions so that it works for me.
I am using a Jetty based server for the response side.
Thanks,
Pankaj Arora
Re: Close_Wait problem
Posted by Ortwin Glück <od...@odi.ch>.
Pankaj Arora wrote:
> Hi,
>
> I am using Http Commons to implement my http client and observe that
> after running the client for large number of request connections are
> left in close_wait stage. I searched through archives and noticed that
> following were suggested:
Pankaj,
Which OS are you using?
First of all, this is perfectly normal and necessary to ensure proper
working of the TCP protocol. The points you listed all assume that
connections are not being closed. But sockets in CLOSE_WAIT *have* been
closed. They are waiting for the other end to complete the TCP FIN
handshake. Depending on your OS this timeout can be largish and can be a
problem if the other side misbehaves or is inappropriately firewalled
(cut connections). Your *only* chance to solve this is on the OS level.
Please read the appropriate documentation for your OS.
Odi
--
[web] http://www.odi.ch/
[blog] http://www.odi.ch/weblog/
[pgp] key 0x81CF3416
finger print F2B1 B21F F056 D53E 5D79 A5AF 02BE 70F5 81CF 3416
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
RE: Close_Wait problem
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, 2007-11-14 at 15:04 -0800, Pankaj Arora wrote:
> Hi,
> It looks like there is some confusion.
> I am doing HttpClient client = getClientInstance();
> Client.exceuteMethod()l
> So each time getClientInsatnce is called closeIdleconnections(20*1000) will be called.
That means idle connections stay in the CLOSE_WAIT state all the time
until #getClientInsatnce() is invoked. You may actually want to close
idle connections after having completed a unit of work not before.
> Also request evry 1 minute is one use case I am facing problem with. I have many more use cases.
>
> Last, I think closeIdleConnections don't close the idle Connections which are not registered with Idle Connection Handler.
>
All connections managed by the MultiThreadedHttpConnectionManager are
supposed to get registered with the idle connection handler
automatically.
> I thought calling multithreadedconnectionmanger.closeidleConnections(20 * 1000) will close all the idle connections which were idle for more than 2-0 sec. but looks like you have to register connection with idleConnectionhandler for it to be closed.
>
>
> Am I right?
>
Not quite.
Oleg
> Thanks,
> Pankaj Arora
>
>
>
> -----Original Message-----
> From: Ortwin Glück [mailto:odi@odi.ch]
> Sent: Wednesday, November 14, 2007 2:17 PM
> To: HttpComponents Project
> Subject: Re: Close_Wait problem
>
>
>
> Pankaj Arora wrote:
> > Hi,
> > I am using a single instance of Http Client. I have implemented using a singleton and each time a method is invoked something like this happens.
> >
> > public static HttpClient getClientInstance(){
> > HttpSingletonHolder.connectionManager.closeIdleConnections(20 * 1000);
> > return HttpSingletonHolder.client;
> > }
> >
> > Client.executeMethod();
> >
> >
> > I was still facing close_Wait problem. So I added
> > public void releaseConnection(HttpConnection conn) {
> >
> > conn.close();
> > super.releaseConnection(conn);
> > }
> >
> > in my MultithreadedConnectionManager.
> >
> > Strangely still see close_wait on Http. My use cases involves invoking server every one minute in a while loop and problem happens every 14-15 hrs.
> >
> > Any suggestions?
> >
> > Thanks,
> > Pankaj Arora
>
> Pankaj,
>
> As I said, there is no active eviction. That means the idle connection are
> closed when and only when you call closeIdleConnections(). You should do this
> periodically in a Thread. In pseudo code that is:
>
> new Thread() {
> public void run() {
> while (!Thread.interrupted()) {
> HttpSingletonHolder.connectionManager.closeIdleConnections(20 * 1000);
> Thread.sleep(20*1000);
> }
> }
>
> }.start();
>
> If you are only making one call every minute I don't see why you are using the
> MultithreadedConnectionManager at all. Just create a new local HttpClient
> instance each time, that's it. There is no point in keeping the instance around
> in a singleton if you use it so rarely.
>
> Odi
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
RE: Close_Wait problem
Posted by Pankaj Arora <pa...@castiron.com>.
Hi,
It looks like there is some confusion.
I am doing HttpClient client = getClientInstance();
Client.exceuteMethod()l
So each time getClientInsatnce is called closeIdleconnections(20*1000) will be called.
Also request evry 1 minute is one use case I am facing problem with. I have many more use cases.
Last, I think closeIdleConnections don't close the idle Connections which are not registered with Idle Connection Handler.
I thought calling multithreadedconnectionmanger.closeidleConnections(20 * 1000) will close all the idle connections which were idle for more than 2-0 sec. but looks like you have to register connection with idleConnectionhandler for it to be closed.
Am I right?
Thanks,
Pankaj Arora
-----Original Message-----
From: Ortwin Glück [mailto:odi@odi.ch]
Sent: Wednesday, November 14, 2007 2:17 PM
To: HttpComponents Project
Subject: Re: Close_Wait problem
Pankaj Arora wrote:
> Hi,
> I am using a single instance of Http Client. I have implemented using a singleton and each time a method is invoked something like this happens.
>
> public static HttpClient getClientInstance(){
> HttpSingletonHolder.connectionManager.closeIdleConnections(20 * 1000);
> return HttpSingletonHolder.client;
> }
>
> Client.executeMethod();
>
>
> I was still facing close_Wait problem. So I added
> public void releaseConnection(HttpConnection conn) {
>
> conn.close();
> super.releaseConnection(conn);
> }
>
> in my MultithreadedConnectionManager.
>
> Strangely still see close_wait on Http. My use cases involves invoking server every one minute in a while loop and problem happens every 14-15 hrs.
>
> Any suggestions?
>
> Thanks,
> Pankaj Arora
Pankaj,
As I said, there is no active eviction. That means the idle connection are
closed when and only when you call closeIdleConnections(). You should do this
periodically in a Thread. In pseudo code that is:
new Thread() {
public void run() {
while (!Thread.interrupted()) {
HttpSingletonHolder.connectionManager.closeIdleConnections(20 * 1000);
Thread.sleep(20*1000);
}
}
}.start();
If you are only making one call every minute I don't see why you are using the
MultithreadedConnectionManager at all. Just create a new local HttpClient
instance each time, that's it. There is no point in keeping the instance around
in a singleton if you use it so rarely.
Odi
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
Re: Close_Wait problem
Posted by Ortwin Glück <od...@odi.ch>.
Pankaj Arora wrote:
> Hi,
> I am using a single instance of Http Client. I have implemented using a singleton and each time a method is invoked something like this happens.
>
> public static HttpClient getClientInstance(){
> HttpSingletonHolder.connectionManager.closeIdleConnections(20 * 1000);
> return HttpSingletonHolder.client;
> }
>
> Client.executeMethod();
>
>
> I was still facing close_Wait problem. So I added
> public void releaseConnection(HttpConnection conn) {
>
> conn.close();
> super.releaseConnection(conn);
> }
>
> in my MultithreadedConnectionManager.
>
> Strangely still see close_wait on Http. My use cases involves invoking server every one minute in a while loop and problem happens every 14-15 hrs.
>
> Any suggestions?
>
> Thanks,
> Pankaj Arora
Pankaj,
As I said, there is no active eviction. That means the idle connection are
closed when and only when you call closeIdleConnections(). You should do this
periodically in a Thread. In pseudo code that is:
new Thread() {
public void run() {
while (!Thread.interrupted()) {
HttpSingletonHolder.connectionManager.closeIdleConnections(20 * 1000);
Thread.sleep(20*1000);
}
}
}.start();
If you are only making one call every minute I don't see why you are using the
MultithreadedConnectionManager at all. Just create a new local HttpClient
instance each time, that's it. There is no point in keeping the instance around
in a singleton if you use it so rarely.
Odi
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
RE: Close_Wait problem
Posted by Pankaj Arora <pa...@castiron.com>.
Hi,
I am using a single instance of Http Client. I have implemented using a singleton and each time a method is invoked something like this happens.
public static HttpClient getClientInstance(){
HttpSingletonHolder.connectionManager.closeIdleConnections(20 * 1000);
return HttpSingletonHolder.client;
}
Client.executeMethod();
I was still facing close_Wait problem. So I added
public void releaseConnection(HttpConnection conn) {
conn.close();
super.releaseConnection(conn);
}
in my MultithreadedConnectionManager.
Strangely still see close_wait on Http. My use cases involves invoking server every one minute in a while loop and problem happens every 14-15 hrs.
Any suggestions?
Thanks,
Pankaj Arora
-----Original Message-----
From: Ortwin Glück [mailto:odi@odi.ch]
Sent: Tuesday, November 13, 2007 1:38 AM
To: HttpComponents Project
Subject: Re: Close_Wait problem
Pankaj,
Okay, I may not be correct here and there actually is something you can do:
Please note that the MultiThreadedConnectionManager has no active
eviction. The pool may fill with stale connections, and depending on
your workload the pool may never have to evict them (with deleteConnection).
You can create an eviction thread which periodically (every 5 seconds
for instance) calls closeIdleConnections(). This should prevent the pool
from filling with stale connections.
As this topic arises every couple of months, and the web is full of
problem reports about CLOSE_WAIT, I think we should add this to the
documentation.
Odi
Pankaj Arora wrote:
> Hi,
>
> I am using Http Commons to implement my http client and observe that
> after running the client for large number of request connections are
> left in close_wait stage. I searched through archives and noticed that
> following were suggested:
>
>
>
> 1. Reuse instances of HttpClient or the associated
> HttpConnectionManager, this may have threading implications.
>
>
> I do that.
>
> 2. Do closeidleConnections(long timeout) or customIdleConnectionTimeout
> thread
>
> I do closeIdleConnections(20*1000). Please do tell if anything else
> needs to be done for this solution.
>
> 3. Add a "Connection: close" and/or "Proxy-Connection: close" header to
>
> all methods. This should work in most cases, assuming that the server
> echoes the header in the response.
>
>
> I don't want to do this as can't take the performance hit.
>
>
> Is there any other way or do I need to do anything else in first 2
> solutions so that it works for me.
>
> I am using a Jetty based server for the response side.
>
> Thanks,
> Pankaj Arora
>
>
>
>
>
>
--
[web] http://www.odi.ch/
[blog] http://www.odi.ch/weblog/
[pgp] key 0x81CF3416
finger print F2B1 B21F F056 D53E 5D79 A5AF 02BE 70F5 81CF 3416
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
Re: Close_Wait problem
Posted by Ortwin Glück <od...@odi.ch>.
Pankaj,
Okay, I may not be correct here and there actually is something you can do:
Please note that the MultiThreadedConnectionManager has no active
eviction. The pool may fill with stale connections, and depending on
your workload the pool may never have to evict them (with deleteConnection).
You can create an eviction thread which periodically (every 5 seconds
for instance) calls closeIdleConnections(). This should prevent the pool
from filling with stale connections.
As this topic arises every couple of months, and the web is full of
problem reports about CLOSE_WAIT, I think we should add this to the
documentation.
Odi
Pankaj Arora wrote:
> Hi,
>
> I am using Http Commons to implement my http client and observe that
> after running the client for large number of request connections are
> left in close_wait stage. I searched through archives and noticed that
> following were suggested:
>
>
>
> 1. Reuse instances of HttpClient or the associated
> HttpConnectionManager, this may have threading implications.
>
>
> I do that.
>
> 2. Do closeidleConnections(long timeout) or customIdleConnectionTimeout
> thread
>
> I do closeIdleConnections(20*1000). Please do tell if anything else
> needs to be done for this solution.
>
> 3. Add a "Connection: close" and/or "Proxy-Connection: close" header to
>
> all methods. This should work in most cases, assuming that the server
> echoes the header in the response.
>
>
> I don't want to do this as can't take the performance hit.
>
>
> Is there any other way or do I need to do anything else in first 2
> solutions so that it works for me.
>
> I am using a Jetty based server for the response side.
>
> Thanks,
> Pankaj Arora
>
>
>
>
>
>
--
[web] http://www.odi.ch/
[blog] http://www.odi.ch/weblog/
[pgp] key 0x81CF3416
finger print F2B1 B21F F056 D53E 5D79 A5AF 02BE 70F5 81CF 3416
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
Re: Close_Wait problem
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Mon, 2007-11-12 at 13:43 -0800, Pankaj Arora wrote:
> Hi,
>
> I am using Http Commons to implement my http client and observe that
> after running the client for large number of request connections are
> left in close_wait stage. I searched through archives and noticed that
> following were suggested:
>
>
>
> 1. Reuse instances of HttpClient or the associated
> HttpConnectionManager, this may have threading implications.
>
>
> I do that.
>
> 2. Do closeidleConnections(long timeout) or customIdleConnectionTimeout
> thread
>
> I do closeIdleConnections(20*1000). Please do tell if anything else
> needs to be done for this solution.
>
You may want to close idle connections more aggressively or simply close
all idle connections (#closeIdleConnections(0)) if you expect a long
period of inactivity.
Hope this helps
Oleg
> 3. Add a "Connection: close" and/or "Proxy-Connection: close" header to
>
> all methods. This should work in most cases, assuming that the server
> echoes the header in the response.
>
>
> I don't want to do this as can't take the performance hit.
>
>
> Is there any other way or do I need to do anything else in first 2
> solutions so that it works for me.
>
> I am using a Jetty based server for the response side.
>
> Thanks,
> Pankaj Arora
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org