You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@axis.apache.org by h b <hb...@gmail.com> on 2008/02/09 00:57:41 UTC

AXIS TCP - TIME_WAIT - Connection Leak

Hello, I'm using a stub that is connecting the server using TCP transport.
I'm looping around calling the service operation.

After I'm done I run netstat -na and I see a whole lot of connections in
TIME_WAIT.  Can I instruct the stub to clean up the connection after a
method call?

Thanks

Re: AXIS TCP - TIME_WAIT - Connection Leak

Posted by Michele Mazzucco <Mi...@ncl.ac.uk>.
On 10 Feb 2008, at 14:09, Andreas Veithen wrote:

> More precisely, the given combination of (local address, local  
> port, remote address, remote port) cannot be reused. Since local  
> address, remote address and remote port are fixed this still leaves  
> more than 60000 possible combinations. Assuming that connections  
> remain in state TIME_WAIT for 240 seconds, you would have to open  
> (and close) connections every 4 milliseconds to hit the limit. Some  
> systems by default don't use the full available port range, so the  
> figures might be a bit different, but anyway, at this connection  
> rate the TCP transport seems not very appropriate and should be  
> replaced by something more sophisticated like HTTP (with keepalive)  
> or JMS.
>
> Maybe it is useful to come back to the initial question asked in  
> this thread:
>
> "Hello, I'm using a stub that is connecting the server using TCP  
> transport.  I'm looping around calling the service operation.
> After I'm done I run netstat -na and I see a whole lot of  
> connections in TIME_WAIT.  Can I instruct the stub to clean up the  
> connection after a method call?"
>
> The complete answer to this question is as follows:
> 1) It is normal to see connections in state TIME_WAIT. This is not  
> an indication of failure of the client to clean up connections, but  
> is required by the TCP protocol.
> 2) Having large numbers of connections in state TIME_WAIT is not a  
> problem in itself.
> 3) If a single client opens and closes more than about 100  
> connections per second, this leads to exhaustion of local ports on  
> the client side. If this is the case, consider using a different  
> transport.


A possible solution to problem #3 would be to customize some kernel  
settings (/etc/sysctl.conf), such as

net.ipv4.tcp_fin_ack_timeout
net.ipv4.tcp_keepalive_time


Michele

>
> Regards,
>
> Andreas
>
>
> On 10 Feb 2008, at 12:18, Michele Mazzucco wrote:
>
>> Andreas,
>>
>> the point is that while the connection is in TIME_WAIT state the  
>> socket cannot be reused (see the links below).
>>
>> Michele
>>
>>
>> http://www.port80software.com/200ok/archive/2004/12/07/205.aspx
>> http://msdn2.microsoft.com/en-us/library/aa560610.aspx
>>
>>
>> On 9 Feb 2008, at 20:59, Andreas Veithen wrote:
>>
>>>
>>> On 09 Feb 2008, at 19:52, Michele Mazzucco wrote:
>>>
>>>>> This is not a problem, since in this state, no resources (other  
>>>>> than the corresponding entries in some internal OS data  
>>>>> structure and that you can see with netstat) are consumed  
>>>>> whatsoever.
>>>>
>>>> This might be a problem, since they increase the number of open  
>>>> file (& connections). Remember that by default linux allows only  
>>>> 1024 open files.
>>>
>>> A connection in state TIME_WAIT is closed (no further packets  
>>> will be sent and no packets except "wandering" packets are  
>>> expected to be received). Therefore there is no file descriptor  
>>> associated to it.
>>>
>>> Andreas
>>>
>>> -------------------------------------------------------------------- 
>>> -
>>> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
>>> For additional commands, e-mail: axis-user-help@ws.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
>> For additional commands, e-mail: axis-user-help@ws.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org


Re: AXIS TCP - TIME_WAIT - Connection Leak

Posted by Andreas Veithen <an...@skynet.be>.
More precisely, the given combination of (local address, local port,  
remote address, remote port) cannot be reused. Since local address,  
remote address and remote port are fixed this still leaves more than  
60000 possible combinations. Assuming that connections remain in state  
TIME_WAIT for 240 seconds, you would have to open (and close)  
connections every 4 milliseconds to hit the limit. Some systems by  
default don't use the full available port range, so the figures might  
be a bit different, but anyway, at this connection rate the TCP  
transport seems not very appropriate and should be replaced by  
something more sophisticated like HTTP (with keepalive) or JMS.

Maybe it is useful to come back to the initial question asked in this  
thread:

"Hello, I'm using a stub that is connecting the server using TCP  
transport.  I'm looping around calling the service operation.
After I'm done I run netstat -na and I see a whole lot of connections  
in TIME_WAIT.  Can I instruct the stub to clean up the connection  
after a method call?"

The complete answer to this question is as follows:
1) It is normal to see connections in state TIME_WAIT. This is not an  
indication of failure of the client to clean up connections, but is  
required by the TCP protocol.
2) Having large numbers of connections in state TIME_WAIT is not a  
problem in itself.
3) If a single client opens and closes more than about 100 connections  
per second, this leads to exhaustion of local ports on the client  
side. If this is the case, consider using a different transport.

Regards,

Andreas


On 10 Feb 2008, at 12:18, Michele Mazzucco wrote:

> Andreas,
>
> the point is that while the connection is in TIME_WAIT state the  
> socket cannot be reused (see the links below).
>
> Michele
>
>
> http://www.port80software.com/200ok/archive/2004/12/07/205.aspx
> http://msdn2.microsoft.com/en-us/library/aa560610.aspx
>
>
> On 9 Feb 2008, at 20:59, Andreas Veithen wrote:
>
>>
>> On 09 Feb 2008, at 19:52, Michele Mazzucco wrote:
>>
>>>> This is not a problem, since in this state, no resources (other  
>>>> than the corresponding entries in some internal OS data structure  
>>>> and that you can see with netstat) are consumed whatsoever.
>>>
>>> This might be a problem, since they increase the number of open  
>>> file (& connections). Remember that by default linux allows only  
>>> 1024 open files.
>>
>> A connection in state TIME_WAIT is closed (no further packets will  
>> be sent and no packets except "wandering" packets are expected to  
>> be received). Therefore there is no file descriptor associated to it.
>>
>> Andreas
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
>> For additional commands, e-mail: axis-user-help@ws.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org


Re: AXIS TCP - TIME_WAIT - Connection Leak

Posted by Michele Mazzucco <Mi...@ncl.ac.uk>.
Andreas,

the point is that while the connection is in TIME_WAIT state the  
socket cannot be reused (see the links below).

Michele


http://www.port80software.com/200ok/archive/2004/12/07/205.aspx
http://msdn2.microsoft.com/en-us/library/aa560610.aspx


On 9 Feb 2008, at 20:59, Andreas Veithen wrote:

>
> On 09 Feb 2008, at 19:52, Michele Mazzucco wrote:
>
>>> This is not a problem, since in this state, no resources (other  
>>> than the corresponding entries in some internal OS data structure  
>>> and that you can see with netstat) are consumed whatsoever.
>>
>> This might be a problem, since they increase the number of open  
>> file (& connections). Remember that by default linux allows only  
>> 1024 open files.
>
> A connection in state TIME_WAIT is closed (no further packets will  
> be sent and no packets except "wandering" packets are expected to  
> be received). Therefore there is no file descriptor associated to it.
>
> Andreas
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org


Re: AXIS TCP - TIME_WAIT - Connection Leak

Posted by Andreas Veithen <an...@skynet.be>.
On 09 Feb 2008, at 19:52, Michele Mazzucco wrote:

>> This is not a problem, since in this state, no resources (other  
>> than the corresponding entries in some internal OS data structure  
>> and that you can see with netstat) are consumed whatsoever.
>
> This might be a problem, since they increase the number of open file  
> (& connections). Remember that by default linux allows only 1024  
> open files.

A connection in state TIME_WAIT is closed (no further packets will be  
sent and no packets except "wandering" packets are expected to be  
received). Therefore there is no file descriptor associated to it.

Andreas

---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org


Re: AXIS TCP - TIME_WAIT - Connection Leak

Posted by Michele Mazzucco <Mi...@ncl.ac.uk>.
On 9 Feb 2008, at 15:48, Andreas Veithen wrote:

> This is not a problem, since in this state, no resources (other  
> than the corresponding entries in some internal OS data structure  
> and that you can see with netstat) are consumed whatsoever.

This might be a problem, since they increase the number of open file  
(& connections). Remember that by default linux allows only 1024 open  
files.


Michele


---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org


Re: AXIS TCP - TIME_WAIT - Connection Leak

Posted by Andreas Veithen <an...@skynet.be>.
A TCP connection goes to state TIME_WAIT after the local end has been  
closed in an orderly way (that is not with a connection reset). On a  
system that opens and closes many connections, it is therefore normal  
to see many connections in this state. This is not a problem, since in  
this state, no resources (other than the corresponding entries in some  
internal OS data structure and that you can see with netstat) are  
consumed whatsoever.

I don't know the exact effect of setCallTransportCleanup(true), but it  
may potentially produce even more connections in state TIME_WAIT  
because connections are not reused.

Andreas

On 09 Feb 2008, at 10:41, Michele Mazzucco wrote:

>
> On 8 Feb 2008, at 23:57, h b wrote:
>
>> Hello, I'm using a stub that is connecting the server using TCP  
>> transport.  I'm looping around calling the service operation.
>>
>> After I'm done I run netstat -na and I see a whole lot of  
>> connections in TIME_WAIT.  Can I instruct the stub to clean up the  
>> connection after a method call?
>
> Try options.setCallTransportCleanup(true)
>
>
> Michele
>>
>> Thanks
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org


Re: AXIS TCP - TIME_WAIT - Connection Leak

Posted by Michele Mazzucco <Mi...@ncl.ac.uk>.
On 8 Feb 2008, at 23:57, h b wrote:

> Hello, I'm using a stub that is connecting the server using TCP  
> transport.  I'm looping around calling the service operation.
>
> After I'm done I run netstat -na and I see a whole lot of  
> connections in TIME_WAIT.  Can I instruct the stub to clean up the  
> connection after a method call?

Try options.setCallTransportCleanup(true)


Michele
>
> Thanks


---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org