CLOSE_WAIT

[Bw57899 <bw...@gmail.com>]

Install an application in apache tomcat (6.0.29) in dev env on Solaris 10 with no issue.

But after move to production, there are always about 50 ~ 100 CLOSE_WAIT on port 1521. The application need connect an Oracle database which is on another server.

So what can I do to check the problem?

Regards
Bill
>> 
>>> 
>>> 
>>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: CLOSE_WAIT

[André Warnier <aw...@ice-sa.com>]

Peter Crowther wrote:
> On 2 February 2011 10:24, Bw57899 <bw...@gmail.com> wrote:
> 
>> Install an application in apache tomcat (6.0.29) in dev env on Solaris 10
>> with no issue.
>>
>> But after move to production, there are always about 50 ~ 100 CLOSE_WAIT on
>> port 1521. The application need connect an Oracle database which is on
>> another server.
>>
>> So what can I do to check the problem?
>>
>> CLOSE_WAIT is normal behaviour - after a TCP socket is closed, it's
> tombstoned for a period so that the TCP stack knows what to do with incoming
> datagrams that might be late.
> 
Peter,
I do not think that this is true, and I believe that you are confusing this with the 
TIME_WAIT state.
See for example this :
http://support.microsoft.com/kb/137984

CLOSE_WAIT:
A socket application has been terminated, but Netstat reports the socket in a CLOSE_WAIT 
state. This could indicate that the client properly closed the connection (FIN has been 
sent), but the server still has its socket open. This could be the result of one instance 
(among all threads or processes) of the socket not being closed.


I had/have a case like that with a third-party Tomcat application.
It goes typically like this :

The webapp creates an object C which among other things makes a TCP connection to another 
server.
The webapp then uses this object's methods to send/receive data from the other server.
At the end of this exchange, the webapp sends a command to the external server, to tell it 
"I'm done".  The external server then closes its end of the connection.
Now the webapp, by means of closing the connection, sets the object C = null.
For the webapp, this means that the connection object C is now effectively "closed".
But in fact, the object C still exists somewhere on the heap, and it still holds on to its 
underlying (OS-level) socket, which has never been closed from the Tomcat server side.
The underlying TCP connection is in the CLOSE_WAIT state, because the socket has never 
been closed on the Tomcat server side, and it remains "dangling".
It only disappears when the Tomcat JVM does a GC, and the Object C is really discarded.
That really closes the underlying TCP socket, and then the state progresses LAST_ACK and 
finally CLOSED and gone.

An easy way to verify if this is the case of the OP, is to force Tomcat to do a GC, and 
see if these CLOSE_WAIT connections then disappear.

If it is the case, then I would advise the OP to check his webapp, to see if it does not 
do the same kind of thing as described above.

One problem that I have seen happen with this, is that as the number of CLOSE_WAIT sockets 
increases (to a few hundred), the whole server becomes unable to handle further TCP 
connections of any kind, being in the practice paralysed.
I suppose that there must exist some inherent limit as to the maximum number of sockets 
which a system (or a process) can have active at any one time.





---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: CLOSE_WAIT

[Peter Crowther <pe...@melandra.com>]

On 2 February 2011 10:24, Bw57899 <bw...@gmail.com> wrote:

> Install an application in apache tomcat (6.0.29) in dev env on Solaris 10
> with no issue.
>
> But after move to production, there are always about 50 ~ 100 CLOSE_WAIT on
> port 1521. The application need connect an Oracle database which is on
> another server.
>
> So what can I do to check the problem?
>
> CLOSE_WAIT is normal behaviour - after a TCP socket is closed, it's
tombstoned for a period so that the TCP stack knows what to do with incoming
datagrams that might be late.

Why do you think this is a problem?  Except that you might be opening and
closing a lot of connections to Oracle?

- Peter

Re: CLOSE_WAIT

[Michael Ludwig <mi...@gmx.de>]

Peter Crowther schrieb am 03.02.2011 um 11:47 (+0000):
> On 3 February 2011 11:35, Pid <pi...@pidster.com> wrote:
> 
> > What factor caused so many people to hijack this thread?
> >
> Using a mail client such as Gmail, which performs its own threading
> and doesn't respect or even show the thread ID.

Or something "Produced By Microsoft Exchange V6.5" (thread hijacker #1),
or Yahoo Mail (TH2), or iPhone (TH3). It's not simply to blame on
technology, I guess. :-)

-- 
Michael Ludwig

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: CLOSE_WAIT

[Peter Crowther <pe...@melandra.com>]

On 3 February 2011 11:35, Pid <pi...@pidster.com> wrote:

> What factor caused so many people to hijack this thread?
>
> Using a mail client such as Gmail, which performs its own threading and
doesn't respect or even show the thread ID.

(And, Andre, you're right, I was confusing the two states - my bad)

- Peter

Re: CLOSE_WAIT

[Pid <pi...@pidster.com>]

What factor caused so many people to hijack this thread?


Dear Bw57899,

Please, in future, do not edit the subject line & body of a reply to an
existing thread, create a new email and address it to
users@tomcat.apache.org - otherwise you leave all of the mail headers
intact.

This frowned-upon practice is called thread-hijacking because it places
new messages in the middle of an existing thread.


p