You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Prashant Deva <pr...@gmail.com> on 2014/06/16 18:57:52 UTC
Tomcat opening thousands of files
I am using Tomcat 7.0.53 with Spring 4 websockets (spring's SockJS support).
I notice that tomcat opens thousands of files.
Just running for a few hours I noticed, it opened 16,552 files:
root@dripcast:~# lsof -p 22691 | wc -l
16552
Our logs are full of this exception thrown tons of times every second:
org.apache.tomcat.util.net.AprEndpoint$Acceptor run
SEVERE: Socket accept failed
org.apache.tomcat.jni.Error: 24: Too many open files
at org.apache.tomcat.jni.Socket.accept(Native Method)
at
org.apache.tomcat.util.net.AprEndpoint$Acceptor.run(AprEndpoint.java:992)
at java.lang.Thread.run(Thread.java:745)
Here is a sample of some of the open files, using lsof.
What can we do to fix this?
java 2056 root *904u sock 0,7 0t0 77846 can't
identify protocol
java 2056 root *905u IPv6 76695 0t0 TCP
dripcast:https->pppoe.95-55-165-198.dynamic.avangarddsl.ru:1376
(ESTABLISHED)
java 2056 root *906u IPv6 74369 0t0 TCP
dripcast:https->fibre.backbonetechnology.com:46415 (ESTABLISHED)
java 2056 root *907u IPv6 76696 0t0 TCP
dripcast:https->outbound.kenshoo.com:14292 (ESTABLISHED)
java 2056 root *908u sock 0,7 0t0 76697 can't
identify protocol
java 2056 root *909u sock 0,7 0t0 76698 can't
identify protocol
java 2056 root *910u 0000 0,9 0 7351 anon_inode
java 2056 root *911u 0000 0,9 0 7351 anon_inode
java 2056 root *912u 0000 0,9 0 7351 anon_inode
java 2056 root *913u 0000 0,9 0 7351 anon_inode
java 2056 root *914u 0000 0,9 0 7351 anon_inode
java 2056 root *915u sock 0,7 0t0 76699 can't
identify protocol
java 2056 root *916u sock 0,7 0t0 76700 can't
identify protocol
java 2056 root *917u sock 0,7 0t0 76701 can't
identify protocol
java 2056 root *918u sock 0,7 0t0 76702 can't
identify protocol
java 2056 root *919u sock 0,7 0t0 77845 can't
identify protocol
java 2056 root *920u sock 0,7 0t0 76705 can't
identify protocol
java 2056 root *921u IPv6 76906 0t0 TCP
dripcast:https->c-67-185-79-130.hsd1.wa.comcast.net:61280 (ESTABLISHED)
java 2056 root *922u 0000 0,9 0 7351 anon_inode
java 2056 root *923u sock 0,7 0t0 77847 can't
identify protocol
java 2056 root *924u sock 0,7 0t0 75196 can't
identify protocol
java 2056 root *925u sock 0,7 0t0 76907 can't
identify protocol
java 2056 root *926u IPv6 76908 0t0 TCP
dripcast:https->cdm-75-108-116-63-nixa.mid.suddenlink.net:56996
(ESTABLISHED)
java 2056 root *927u 0000 0,9 0 7351 anon_inode
java 2056 root *928u IPv6 76909 0t0 TCP
dripcast:https->fibre.backbonetechnology.com:46416 (ESTABLISHED)
java 2056 root *929u sock 0,7 0t0 76910 can't
identify protocol
java 2056 root *930u sock 0,7 0t0 76911 can't
identify protocol
java 2056 root *931u IPv6 76912 0t0 TCP
dripcast:https->outbound.kenshoo.com:14309 (ESTABLISHED)
java 2056 root *932u 0000 0,9 0 7351 anon_inode
java 2056 root *933u 0000 0,9 0 7351 anon_inode
java 2056 root *934u 0000 0,9 0 7351 anon_inode
java 2056 root *935u 0000 0,9 0 7351 anon_inode
java 2056 root *936u 0000 0,9 0 7351 anon_inode
java 2056 root *937u sock 0,7 0t0 73405 can't
identify protocol
java 2056 root *938u sock 0,7 0t0 73406 can't
identify protocol
java 2056 root *939u sock 0,7 0t0 73407 can't
identify protocol
java 2056 root *940u sock 0,7 0t0 71475 can't
identify protocol
java 2056 root *941u sock 0,7 0t0 74371 can't
identify protocol
java 2056 root *942u 0000 0,9 0 7351 anon_inode
java 2056 root *943u sock 0,7 0t0 74372 can't
identify protocol
java 2056 root *944u sock 0,7 0t0 74373 can't
identify protocol
java 2056 root *945u sock 0,7 0t0 74378 can't
identify protocol
java 2056 root *946u 0000 0,9 0 7351 anon_inode
java 2056 root *947u sock 0,7 0t0 71476 can't
identify protocol
java 2056 root *948u sock 0,7 0t0 74379 can't
identify protocol
java 2056 root *949u sock 0,7 0t0 76929 can't
identify protocol
java 2056 root *950u 0000 0,9 0 7351 anon_inode
java 2056 root *951u IPv6 76922 0t0 TCP
dripcast:https->c-67-185-79-130.hsd1.wa.comcast.net:61281 (ESTABLISHED)
java 2056 root *952u sock 0,7 0t0 76923 can't
identify protocol
java 2056 root *953u sock 0,7 0t0 76930 can't
identify protocol
java 2056 root *954u IPv6 76924 0t0 TCP
dripcast:https->outbound.kenshoo.com:14337 (ESTABLISHED)
java 2056 root *955u sock 0,7 0t0 76925 can't
identify protocol
java 2056 root *956u IPv6 76926 0t0 TCP
dripcast:https->kpersfw.state.ks.us:62427 (CLOSE_WAIT)
java 2056 root *957u sock 0,7 0t0 76931 can't
identify protocol
java 2056 root *958u 0000 0,9 0 7351 anon_inode
java 2056 root *959u 0000 0,9 0 7351 anon_inode
java 2056 root *960u 0000 0,9 0 7351 anon_inode
java 2056 root *961u 0000 0,9 0 7351 anon_inode
java 2056 root *962u 0000 0,9 0 7351 anon_inode
Prashant
Re: Tomcat opening thousands of files
Posted by Martin Grigorov <mg...@apache.org>.
Hi Prashant,
On Tue, Jun 17, 2014 at 2:40 PM, Prashant Deva <pr...@gmail.com>
wrote:
> ok running on tomcat 8.0.8 with spring 4.0.5 and reactor 1.1.2, my log
> within minutes gets filled with these exceptions.
>
> Seems like it could be a tomcat issue after all:
>
This error could be also read as: the client (the browser) closed the
connection unexpectedly. So it could be an error in the application too
The best you can do is to create a mini application that exposes the
problem and attach it to a ticket in Tomcat's Bugzilla or Spring's JIRA
with detailed explanation how to reproduce the problem.
> 11:35:37,922 ERROR http-nio-80-exec-37
> handler.LoggingWebSocketHandlerDecorator:61 - Transport error for SockJS
> session id=hv4ncrvg
> java.io.IOException: Connection reset by peer
> at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
> at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
> at sun.nio.ch.IOUtil.read(IOUtil.java:197)
> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
> at org.apache.tomcat.util.net.NioChannel.read(NioChannel.java:135)
> at
>
> org.apache.coyote.http11.upgrade.NioServletInputStream.fillReadBuffer(NioServletInputStream.java:136)
> at
>
> org.apache.coyote.http11.upgrade.NioServletInputStream.doRead(NioServletInputStream.java:80)
> at
>
> org.apache.coyote.http11.upgrade.AbstractServletInputStream.read(AbstractServletInputStream.java:120)
> at
>
> org.apache.tomcat.websocket.server.WsFrameServer.onDataAvailable(WsFrameServer.java:46)
> at
>
> org.apache.tomcat.websocket.server.WsHttpUpgradeHandler$WsReadListener.onDataAvailable(WsHttpUpgradeHandler.java:194)
> at
>
> org.apache.coyote.http11.upgrade.AbstractServletInputStream.onDataAvailable(AbstractServletInputStream.java:194)
> at
>
> org.apache.coyote.http11.upgrade.AbstractProcessor.upgradeDispatch(AbstractProcessor.java:95)
> at
>
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:650)
> at
>
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222)
> at
>
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1566)
> at
>
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1523)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
>
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:744)
>
>
> Prashant
>
>
> On Mon, Jun 16, 2014 at 5:26 PM, Prashant Deva <pr...@gmail.com>
> wrote:
>
> > just noticed this, you tried running on tomcat 8, while we are running
> > 7.0.53...
> >
> > Prashant
> >
> >
> > On Mon, Jun 16, 2014 at 1:36 PM, Prashant Deva <pr...@gmail.com>
> > wrote:
> >
> >> our production instance (which we are running into issues with) has
> about
> >> 2500 concurrent users.
> >>
> >> Prashant
> >>
> >>
> >> On Mon, Jun 16, 2014 at 1:26 PM, Rossen Stoyanchev <
> >> rstoyanchev@gopivotal.com> wrote:
> >>
> >>> On Mon, Jun 16, 2014 at 4:04 PM, Prashant Deva <
> prashant.deva@gmail.com>
> >>> wrote:
> >>>
> >>> > Rossen,
> >>> > Did you use an external queue?
> >>>
> >>>
> >>> Yes I did have the sample configured to use RabbitMQ for broadcasting
> >>> messages. That's running as a separate process though so it shouldn't
> >>> change the output of "lsof".
> >>>
> >>> How many clients were connected at the same time?
> >>> >
> >>>
> >>> In the sample, just one. I also ran a load test with 500 concurrent
> users
> >>> (1 million messages) and the file descriptor count remains stable
> (around
> >>> 500).
> >>>
> >>> I'd reverse the question, how many users do you have to run with to
> >>> demonstrate the issue?
> >>>
> >>> Rossen
> >>>
> >>
> >>
> >
>
Re: Tomcat opening thousands of files
Posted by Prashant Deva <pr...@gmail.com>.
ok running on tomcat 8.0.8 with spring 4.0.5 and reactor 1.1.2, my log
within minutes gets filled with these exceptions.
Seems like it could be a tomcat issue after all:
11:35:37,922 ERROR http-nio-80-exec-37
handler.LoggingWebSocketHandlerDecorator:61 - Transport error for SockJS
session id=hv4ncrvg
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:197)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
at org.apache.tomcat.util.net.NioChannel.read(NioChannel.java:135)
at
org.apache.coyote.http11.upgrade.NioServletInputStream.fillReadBuffer(NioServletInputStream.java:136)
at
org.apache.coyote.http11.upgrade.NioServletInputStream.doRead(NioServletInputStream.java:80)
at
org.apache.coyote.http11.upgrade.AbstractServletInputStream.read(AbstractServletInputStream.java:120)
at
org.apache.tomcat.websocket.server.WsFrameServer.onDataAvailable(WsFrameServer.java:46)
at
org.apache.tomcat.websocket.server.WsHttpUpgradeHandler$WsReadListener.onDataAvailable(WsHttpUpgradeHandler.java:194)
at
org.apache.coyote.http11.upgrade.AbstractServletInputStream.onDataAvailable(AbstractServletInputStream.java:194)
at
org.apache.coyote.http11.upgrade.AbstractProcessor.upgradeDispatch(AbstractProcessor.java:95)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:650)
at
org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1566)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1523)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:744)
Prashant
On Mon, Jun 16, 2014 at 5:26 PM, Prashant Deva <pr...@gmail.com>
wrote:
> just noticed this, you tried running on tomcat 8, while we are running
> 7.0.53...
>
> Prashant
>
>
> On Mon, Jun 16, 2014 at 1:36 PM, Prashant Deva <pr...@gmail.com>
> wrote:
>
>> our production instance (which we are running into issues with) has about
>> 2500 concurrent users.
>>
>> Prashant
>>
>>
>> On Mon, Jun 16, 2014 at 1:26 PM, Rossen Stoyanchev <
>> rstoyanchev@gopivotal.com> wrote:
>>
>>> On Mon, Jun 16, 2014 at 4:04 PM, Prashant Deva <pr...@gmail.com>
>>> wrote:
>>>
>>> > Rossen,
>>> > Did you use an external queue?
>>>
>>>
>>> Yes I did have the sample configured to use RabbitMQ for broadcasting
>>> messages. That's running as a separate process though so it shouldn't
>>> change the output of "lsof".
>>>
>>> How many clients were connected at the same time?
>>> >
>>>
>>> In the sample, just one. I also ran a load test with 500 concurrent users
>>> (1 million messages) and the file descriptor count remains stable (around
>>> 500).
>>>
>>> I'd reverse the question, how many users do you have to run with to
>>> demonstrate the issue?
>>>
>>> Rossen
>>>
>>
>>
>
Re: Tomcat opening thousands of files
Posted by Prashant Deva <pr...@gmail.com>.
just noticed this, you tried running on tomcat 8, while we are running
7.0.53...
Prashant
On Mon, Jun 16, 2014 at 1:36 PM, Prashant Deva <pr...@gmail.com>
wrote:
> our production instance (which we are running into issues with) has about
> 2500 concurrent users.
>
> Prashant
>
>
> On Mon, Jun 16, 2014 at 1:26 PM, Rossen Stoyanchev <
> rstoyanchev@gopivotal.com> wrote:
>
>> On Mon, Jun 16, 2014 at 4:04 PM, Prashant Deva <pr...@gmail.com>
>> wrote:
>>
>> > Rossen,
>> > Did you use an external queue?
>>
>>
>> Yes I did have the sample configured to use RabbitMQ for broadcasting
>> messages. That's running as a separate process though so it shouldn't
>> change the output of "lsof".
>>
>> How many clients were connected at the same time?
>> >
>>
>> In the sample, just one. I also ran a load test with 500 concurrent users
>> (1 million messages) and the file descriptor count remains stable (around
>> 500).
>>
>> I'd reverse the question, how many users do you have to run with to
>> demonstrate the issue?
>>
>> Rossen
>>
>
>
Re: Tomcat opening thousands of files
Posted by Prashant Deva <pr...@gmail.com>.
our production instance (which we are running into issues with) has about
2500 concurrent users.
Prashant
On Mon, Jun 16, 2014 at 1:26 PM, Rossen Stoyanchev <
rstoyanchev@gopivotal.com> wrote:
> On Mon, Jun 16, 2014 at 4:04 PM, Prashant Deva <pr...@gmail.com>
> wrote:
>
> > Rossen,
> > Did you use an external queue?
>
>
> Yes I did have the sample configured to use RabbitMQ for broadcasting
> messages. That's running as a separate process though so it shouldn't
> change the output of "lsof".
>
> How many clients were connected at the same time?
> >
>
> In the sample, just one. I also ran a load test with 500 concurrent users
> (1 million messages) and the file descriptor count remains stable (around
> 500).
>
> I'd reverse the question, how many users do you have to run with to
> demonstrate the issue?
>
> Rossen
>
Re: Tomcat opening thousands of files
Posted by Rossen Stoyanchev <rs...@gopivotal.com>.
On Mon, Jun 16, 2014 at 4:04 PM, Prashant Deva <pr...@gmail.com>
wrote:
> Rossen,
> Did you use an external queue?
Yes I did have the sample configured to use RabbitMQ for broadcasting
messages. That's running as a separate process though so it shouldn't
change the output of "lsof".
How many clients were connected at the same time?
>
In the sample, just one. I also ran a load test with 500 concurrent users
(1 million messages) and the file descriptor count remains stable (around
500).
I'd reverse the question, how many users do you have to run with to
demonstrate the issue?
Rossen
Re: Tomcat opening thousands of files
Posted by Prashant Deva <pr...@gmail.com>.
Rossen,
Did you use an external queue? How many clients were connected at the same
time?
Prashant
On Mon, Jun 16, 2014 at 12:02 PM, Rossen Stoyanchev <
rstoyanchev@gopivotal.com> wrote:
> I gave this a quick try with the spring-websocket-portfolio sample [1] with
> Tomcat 8. The output of the lsof command is just over 200 initially and
> then goes up a little but not much (about 10).
>
> From the client-side I configured the SockJS client to use the WebSocket
> transport (i.e. using Tomcat's WebSocket engine on the server side) first
> and then tried the iframe-eventsource transport (SSE) which uses Tomcat's
> Servlet 3 async request support to stream messages over HTTP. No
> difference.
>
> Prashant you'll need to narrow it down a bit to see what is the key to
> reproducing the issue.
>
>
> Rossen
>
>
> [1] https://github.com/rstoyanchev/spring-websocket-portfolio
>
>
> On Mon, Jun 16, 2014 at 1:25 PM, Prashant Deva <pr...@gmail.com>
> wrote:
>
> > hmm, we are just using spring websockets to send messages. In fact there
> is
> > only two 'topics' in total, so there isnt much the application is doing
> > quite frankly.
> > do you think this is a spring issue?
> >
> > Prashant
> >
> >
> > On Mon, Jun 16, 2014 at 10:10 AM, David kerber <dc...@verizon.net>
> > wrote:
> >
> > > On 6/16/2014 12:57 PM, Prashant Deva wrote:
> > >
> > >> I am using Tomcat 7.0.53 with Spring 4 websockets (spring's SockJS
> > >> support).
> > >> I notice that tomcat opens thousands of files.
> > >>
> > >
> > > Correction: your application opens thousands of files, or leaves them
> > > open when it should have closed them.
> > >
> > >
> > >
> > >
> > > Just running for a few hours I noticed, it opened 16,552 files:
> > >>
> > >> root@dripcast:~# lsof -p 22691 | wc -l
> > >> 16552
> > >>
> > >> Our logs are full of this exception thrown tons of times every second:
> > >> org.apache.tomcat.util.net.AprEndpoint$Acceptor run
> > >> SEVERE: Socket accept failed
> > >> org.apache.tomcat.jni.Error: 24: Too many open files
> > >> at org.apache.tomcat.jni.Socket.accept(Native Method)
> > >> at
> > >>
> > org.apache.tomcat.util.net.AprEndpoint$Acceptor.run(AprEndpoint.java:992)
> > >> at java.lang.Thread.run(Thread.java:745)
> > >>
> > >>
> > >> Here is a sample of some of the open files, using lsof.
> > >> What can we do to fix this?
> > >>
> > >> java 2056 root *904u sock 0,7 0t0 77846 can't
> > >> identify protocol
> > >> java 2056 root *905u IPv6 76695 0t0 TCP
> > >> dripcast:https->pppoe.95-55-165-198.dynamic.avangarddsl.ru:1376
> > >> (ESTABLISHED)
> > >> java 2056 root *906u IPv6 74369 0t0 TCP
> > >> dripcast:https->fibre.backbonetechnology.com:46415 (ESTABLISHED)
> > >> java 2056 root *907u IPv6 76696 0t0 TCP
> > >> dripcast:https->outbound.kenshoo.com:14292 (ESTABLISHED)
> > >> java 2056 root *908u sock 0,7 0t0 76697 can't
> > >> identify protocol
> > >> java 2056 root *909u sock 0,7 0t0 76698 can't
> > >> identify protocol
> > >> java 2056 root *910u 0000 0,9 0 7351
> > >> anon_inode
> > >> java 2056 root *911u 0000 0,9 0 7351
> > >> anon_inode
> > >> java 2056 root *912u 0000 0,9 0 7351
> > >> anon_inode
> > >> java 2056 root *913u 0000 0,9 0 7351
> > >> anon_inode
> > >> java 2056 root *914u 0000 0,9 0 7351
> > >> anon_inode
> > >> java 2056 root *915u sock 0,7 0t0 76699 can't
> > >> identify protocol
> > >> java 2056 root *916u sock 0,7 0t0 76700 can't
> > >> identify protocol
> > >> java 2056 root *917u sock 0,7 0t0 76701 can't
> > >> identify protocol
> > >> java 2056 root *918u sock 0,7 0t0 76702 can't
> > >> identify protocol
> > >> java 2056 root *919u sock 0,7 0t0 77845 can't
> > >> identify protocol
> > >> java 2056 root *920u sock 0,7 0t0 76705 can't
> > >> identify protocol
> > >> java 2056 root *921u IPv6 76906 0t0 TCP
> > >> dripcast:https->c-67-185-79-130.hsd1.wa.comcast.net:61280
> (ESTABLISHED)
> > >> java 2056 root *922u 0000 0,9 0 7351
> > >> anon_inode
> > >> java 2056 root *923u sock 0,7 0t0 77847 can't
> > >> identify protocol
> > >> java 2056 root *924u sock 0,7 0t0 75196 can't
> > >> identify protocol
> > >> java 2056 root *925u sock 0,7 0t0 76907 can't
> > >> identify protocol
> > >> java 2056 root *926u IPv6 76908 0t0 TCP
> > >> dripcast:https->cdm-75-108-116-63-nixa.mid.suddenlink.net:56996
> > >> (ESTABLISHED)
> > >> java 2056 root *927u 0000 0,9 0 7351
> > >> anon_inode
> > >> java 2056 root *928u IPv6 76909 0t0 TCP
> > >> dripcast:https->fibre.backbonetechnology.com:46416 (ESTABLISHED)
> > >> java 2056 root *929u sock 0,7 0t0 76910 can't
> > >> identify protocol
> > >> java 2056 root *930u sock 0,7 0t0 76911 can't
> > >> identify protocol
> > >> java 2056 root *931u IPv6 76912 0t0 TCP
> > >> dripcast:https->outbound.kenshoo.com:14309 (ESTABLISHED)
> > >> java 2056 root *932u 0000 0,9 0 7351
> > >> anon_inode
> > >> java 2056 root *933u 0000 0,9 0 7351
> > >> anon_inode
> > >> java 2056 root *934u 0000 0,9 0 7351
> > >> anon_inode
> > >> java 2056 root *935u 0000 0,9 0 7351
> > >> anon_inode
> > >> java 2056 root *936u 0000 0,9 0 7351
> > >> anon_inode
> > >> java 2056 root *937u sock 0,7 0t0 73405 can't
> > >> identify protocol
> > >> java 2056 root *938u sock 0,7 0t0 73406 can't
> > >> identify protocol
> > >> java 2056 root *939u sock 0,7 0t0 73407 can't
> > >> identify protocol
> > >> java 2056 root *940u sock 0,7 0t0 71475 can't
> > >> identify protocol
> > >> java 2056 root *941u sock 0,7 0t0 74371 can't
> > >> identify protocol
> > >> java 2056 root *942u 0000 0,9 0 7351
> > >> anon_inode
> > >> java 2056 root *943u sock 0,7 0t0 74372 can't
> > >> identify protocol
> > >> java 2056 root *944u sock 0,7 0t0 74373 can't
> > >> identify protocol
> > >> java 2056 root *945u sock 0,7 0t0 74378 can't
> > >> identify protocol
> > >> java 2056 root *946u 0000 0,9 0 7351
> > >> anon_inode
> > >> java 2056 root *947u sock 0,7 0t0 71476 can't
> > >> identify protocol
> > >> java 2056 root *948u sock 0,7 0t0 74379 can't
> > >> identify protocol
> > >> java 2056 root *949u sock 0,7 0t0 76929 can't
> > >> identify protocol
> > >> java 2056 root *950u 0000 0,9 0 7351
> > >> anon_inode
> > >> java 2056 root *951u IPv6 76922 0t0 TCP
> > >> dripcast:https->c-67-185-79-130.hsd1.wa.comcast.net:61281
> (ESTABLISHED)
> > >> java 2056 root *952u sock 0,7 0t0 76923 can't
> > >> identify protocol
> > >> java 2056 root *953u sock 0,7 0t0 76930 can't
> > >> identify protocol
> > >> java 2056 root *954u IPv6 76924 0t0 TCP
> > >> dripcast:https->outbound.kenshoo.com:14337 (ESTABLISHED)
> > >> java 2056 root *955u sock 0,7 0t0 76925 can't
> > >> identify protocol
> > >> java 2056 root *956u IPv6 76926 0t0 TCP
> > >> dripcast:https->kpersfw.state.ks.us:62427 (CLOSE_WAIT)
> > >> java 2056 root *957u sock 0,7 0t0 76931 can't
> > >> identify protocol
> > >> java 2056 root *958u 0000 0,9 0 7351
> > >> anon_inode
> > >> java 2056 root *959u 0000 0,9 0 7351
> > >> anon_inode
> > >> java 2056 root *960u 0000 0,9 0 7351
> > >> anon_inode
> > >> java 2056 root *961u 0000 0,9 0 7351
> > >> anon_inode
> > >> java 2056 root *962u 0000 0,9 0 7351
> > >> anon_inode
> > >>
> > >>
> > >> Prashant
> > >>
> > >>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > > For additional commands, e-mail: users-help@tomcat.apache.org
> > >
> > >
> >
>
Re: Tomcat opening thousands of files
Posted by Rossen Stoyanchev <rs...@gopivotal.com>.
I gave this a quick try with the spring-websocket-portfolio sample [1] with
Tomcat 8. The output of the lsof command is just over 200 initially and
then goes up a little but not much (about 10).
>From the client-side I configured the SockJS client to use the WebSocket
transport (i.e. using Tomcat's WebSocket engine on the server side) first
and then tried the iframe-eventsource transport (SSE) which uses Tomcat's
Servlet 3 async request support to stream messages over HTTP. No difference.
Prashant you'll need to narrow it down a bit to see what is the key to
reproducing the issue.
Rossen
[1] https://github.com/rstoyanchev/spring-websocket-portfolio
On Mon, Jun 16, 2014 at 1:25 PM, Prashant Deva <pr...@gmail.com>
wrote:
> hmm, we are just using spring websockets to send messages. In fact there is
> only two 'topics' in total, so there isnt much the application is doing
> quite frankly.
> do you think this is a spring issue?
>
> Prashant
>
>
> On Mon, Jun 16, 2014 at 10:10 AM, David kerber <dc...@verizon.net>
> wrote:
>
> > On 6/16/2014 12:57 PM, Prashant Deva wrote:
> >
> >> I am using Tomcat 7.0.53 with Spring 4 websockets (spring's SockJS
> >> support).
> >> I notice that tomcat opens thousands of files.
> >>
> >
> > Correction: your application opens thousands of files, or leaves them
> > open when it should have closed them.
> >
> >
> >
> >
> > Just running for a few hours I noticed, it opened 16,552 files:
> >>
> >> root@dripcast:~# lsof -p 22691 | wc -l
> >> 16552
> >>
> >> Our logs are full of this exception thrown tons of times every second:
> >> org.apache.tomcat.util.net.AprEndpoint$Acceptor run
> >> SEVERE: Socket accept failed
> >> org.apache.tomcat.jni.Error: 24: Too many open files
> >> at org.apache.tomcat.jni.Socket.accept(Native Method)
> >> at
> >>
> org.apache.tomcat.util.net.AprEndpoint$Acceptor.run(AprEndpoint.java:992)
> >> at java.lang.Thread.run(Thread.java:745)
> >>
> >>
> >> Here is a sample of some of the open files, using lsof.
> >> What can we do to fix this?
> >>
> >> java 2056 root *904u sock 0,7 0t0 77846 can't
> >> identify protocol
> >> java 2056 root *905u IPv6 76695 0t0 TCP
> >> dripcast:https->pppoe.95-55-165-198.dynamic.avangarddsl.ru:1376
> >> (ESTABLISHED)
> >> java 2056 root *906u IPv6 74369 0t0 TCP
> >> dripcast:https->fibre.backbonetechnology.com:46415 (ESTABLISHED)
> >> java 2056 root *907u IPv6 76696 0t0 TCP
> >> dripcast:https->outbound.kenshoo.com:14292 (ESTABLISHED)
> >> java 2056 root *908u sock 0,7 0t0 76697 can't
> >> identify protocol
> >> java 2056 root *909u sock 0,7 0t0 76698 can't
> >> identify protocol
> >> java 2056 root *910u 0000 0,9 0 7351
> >> anon_inode
> >> java 2056 root *911u 0000 0,9 0 7351
> >> anon_inode
> >> java 2056 root *912u 0000 0,9 0 7351
> >> anon_inode
> >> java 2056 root *913u 0000 0,9 0 7351
> >> anon_inode
> >> java 2056 root *914u 0000 0,9 0 7351
> >> anon_inode
> >> java 2056 root *915u sock 0,7 0t0 76699 can't
> >> identify protocol
> >> java 2056 root *916u sock 0,7 0t0 76700 can't
> >> identify protocol
> >> java 2056 root *917u sock 0,7 0t0 76701 can't
> >> identify protocol
> >> java 2056 root *918u sock 0,7 0t0 76702 can't
> >> identify protocol
> >> java 2056 root *919u sock 0,7 0t0 77845 can't
> >> identify protocol
> >> java 2056 root *920u sock 0,7 0t0 76705 can't
> >> identify protocol
> >> java 2056 root *921u IPv6 76906 0t0 TCP
> >> dripcast:https->c-67-185-79-130.hsd1.wa.comcast.net:61280 (ESTABLISHED)
> >> java 2056 root *922u 0000 0,9 0 7351
> >> anon_inode
> >> java 2056 root *923u sock 0,7 0t0 77847 can't
> >> identify protocol
> >> java 2056 root *924u sock 0,7 0t0 75196 can't
> >> identify protocol
> >> java 2056 root *925u sock 0,7 0t0 76907 can't
> >> identify protocol
> >> java 2056 root *926u IPv6 76908 0t0 TCP
> >> dripcast:https->cdm-75-108-116-63-nixa.mid.suddenlink.net:56996
> >> (ESTABLISHED)
> >> java 2056 root *927u 0000 0,9 0 7351
> >> anon_inode
> >> java 2056 root *928u IPv6 76909 0t0 TCP
> >> dripcast:https->fibre.backbonetechnology.com:46416 (ESTABLISHED)
> >> java 2056 root *929u sock 0,7 0t0 76910 can't
> >> identify protocol
> >> java 2056 root *930u sock 0,7 0t0 76911 can't
> >> identify protocol
> >> java 2056 root *931u IPv6 76912 0t0 TCP
> >> dripcast:https->outbound.kenshoo.com:14309 (ESTABLISHED)
> >> java 2056 root *932u 0000 0,9 0 7351
> >> anon_inode
> >> java 2056 root *933u 0000 0,9 0 7351
> >> anon_inode
> >> java 2056 root *934u 0000 0,9 0 7351
> >> anon_inode
> >> java 2056 root *935u 0000 0,9 0 7351
> >> anon_inode
> >> java 2056 root *936u 0000 0,9 0 7351
> >> anon_inode
> >> java 2056 root *937u sock 0,7 0t0 73405 can't
> >> identify protocol
> >> java 2056 root *938u sock 0,7 0t0 73406 can't
> >> identify protocol
> >> java 2056 root *939u sock 0,7 0t0 73407 can't
> >> identify protocol
> >> java 2056 root *940u sock 0,7 0t0 71475 can't
> >> identify protocol
> >> java 2056 root *941u sock 0,7 0t0 74371 can't
> >> identify protocol
> >> java 2056 root *942u 0000 0,9 0 7351
> >> anon_inode
> >> java 2056 root *943u sock 0,7 0t0 74372 can't
> >> identify protocol
> >> java 2056 root *944u sock 0,7 0t0 74373 can't
> >> identify protocol
> >> java 2056 root *945u sock 0,7 0t0 74378 can't
> >> identify protocol
> >> java 2056 root *946u 0000 0,9 0 7351
> >> anon_inode
> >> java 2056 root *947u sock 0,7 0t0 71476 can't
> >> identify protocol
> >> java 2056 root *948u sock 0,7 0t0 74379 can't
> >> identify protocol
> >> java 2056 root *949u sock 0,7 0t0 76929 can't
> >> identify protocol
> >> java 2056 root *950u 0000 0,9 0 7351
> >> anon_inode
> >> java 2056 root *951u IPv6 76922 0t0 TCP
> >> dripcast:https->c-67-185-79-130.hsd1.wa.comcast.net:61281 (ESTABLISHED)
> >> java 2056 root *952u sock 0,7 0t0 76923 can't
> >> identify protocol
> >> java 2056 root *953u sock 0,7 0t0 76930 can't
> >> identify protocol
> >> java 2056 root *954u IPv6 76924 0t0 TCP
> >> dripcast:https->outbound.kenshoo.com:14337 (ESTABLISHED)
> >> java 2056 root *955u sock 0,7 0t0 76925 can't
> >> identify protocol
> >> java 2056 root *956u IPv6 76926 0t0 TCP
> >> dripcast:https->kpersfw.state.ks.us:62427 (CLOSE_WAIT)
> >> java 2056 root *957u sock 0,7 0t0 76931 can't
> >> identify protocol
> >> java 2056 root *958u 0000 0,9 0 7351
> >> anon_inode
> >> java 2056 root *959u 0000 0,9 0 7351
> >> anon_inode
> >> java 2056 root *960u 0000 0,9 0 7351
> >> anon_inode
> >> java 2056 root *961u 0000 0,9 0 7351
> >> anon_inode
> >> java 2056 root *962u 0000 0,9 0 7351
> >> anon_inode
> >>
> >>
> >> Prashant
> >>
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > For additional commands, e-mail: users-help@tomcat.apache.org
> >
> >
>
Re: Tomcat opening thousands of files
Posted by Prashant Deva <pr...@gmail.com>.
hmm, we are just using spring websockets to send messages. In fact there is
only two 'topics' in total, so there isnt much the application is doing
quite frankly.
do you think this is a spring issue?
Prashant
On Mon, Jun 16, 2014 at 10:10 AM, David kerber <dc...@verizon.net> wrote:
> On 6/16/2014 12:57 PM, Prashant Deva wrote:
>
>> I am using Tomcat 7.0.53 with Spring 4 websockets (spring's SockJS
>> support).
>> I notice that tomcat opens thousands of files.
>>
>
> Correction: your application opens thousands of files, or leaves them
> open when it should have closed them.
>
>
>
>
> Just running for a few hours I noticed, it opened 16,552 files:
>>
>> root@dripcast:~# lsof -p 22691 | wc -l
>> 16552
>>
>> Our logs are full of this exception thrown tons of times every second:
>> org.apache.tomcat.util.net.AprEndpoint$Acceptor run
>> SEVERE: Socket accept failed
>> org.apache.tomcat.jni.Error: 24: Too many open files
>> at org.apache.tomcat.jni.Socket.accept(Native Method)
>> at
>> org.apache.tomcat.util.net.AprEndpoint$Acceptor.run(AprEndpoint.java:992)
>> at java.lang.Thread.run(Thread.java:745)
>>
>>
>> Here is a sample of some of the open files, using lsof.
>> What can we do to fix this?
>>
>> java 2056 root *904u sock 0,7 0t0 77846 can't
>> identify protocol
>> java 2056 root *905u IPv6 76695 0t0 TCP
>> dripcast:https->pppoe.95-55-165-198.dynamic.avangarddsl.ru:1376
>> (ESTABLISHED)
>> java 2056 root *906u IPv6 74369 0t0 TCP
>> dripcast:https->fibre.backbonetechnology.com:46415 (ESTABLISHED)
>> java 2056 root *907u IPv6 76696 0t0 TCP
>> dripcast:https->outbound.kenshoo.com:14292 (ESTABLISHED)
>> java 2056 root *908u sock 0,7 0t0 76697 can't
>> identify protocol
>> java 2056 root *909u sock 0,7 0t0 76698 can't
>> identify protocol
>> java 2056 root *910u 0000 0,9 0 7351
>> anon_inode
>> java 2056 root *911u 0000 0,9 0 7351
>> anon_inode
>> java 2056 root *912u 0000 0,9 0 7351
>> anon_inode
>> java 2056 root *913u 0000 0,9 0 7351
>> anon_inode
>> java 2056 root *914u 0000 0,9 0 7351
>> anon_inode
>> java 2056 root *915u sock 0,7 0t0 76699 can't
>> identify protocol
>> java 2056 root *916u sock 0,7 0t0 76700 can't
>> identify protocol
>> java 2056 root *917u sock 0,7 0t0 76701 can't
>> identify protocol
>> java 2056 root *918u sock 0,7 0t0 76702 can't
>> identify protocol
>> java 2056 root *919u sock 0,7 0t0 77845 can't
>> identify protocol
>> java 2056 root *920u sock 0,7 0t0 76705 can't
>> identify protocol
>> java 2056 root *921u IPv6 76906 0t0 TCP
>> dripcast:https->c-67-185-79-130.hsd1.wa.comcast.net:61280 (ESTABLISHED)
>> java 2056 root *922u 0000 0,9 0 7351
>> anon_inode
>> java 2056 root *923u sock 0,7 0t0 77847 can't
>> identify protocol
>> java 2056 root *924u sock 0,7 0t0 75196 can't
>> identify protocol
>> java 2056 root *925u sock 0,7 0t0 76907 can't
>> identify protocol
>> java 2056 root *926u IPv6 76908 0t0 TCP
>> dripcast:https->cdm-75-108-116-63-nixa.mid.suddenlink.net:56996
>> (ESTABLISHED)
>> java 2056 root *927u 0000 0,9 0 7351
>> anon_inode
>> java 2056 root *928u IPv6 76909 0t0 TCP
>> dripcast:https->fibre.backbonetechnology.com:46416 (ESTABLISHED)
>> java 2056 root *929u sock 0,7 0t0 76910 can't
>> identify protocol
>> java 2056 root *930u sock 0,7 0t0 76911 can't
>> identify protocol
>> java 2056 root *931u IPv6 76912 0t0 TCP
>> dripcast:https->outbound.kenshoo.com:14309 (ESTABLISHED)
>> java 2056 root *932u 0000 0,9 0 7351
>> anon_inode
>> java 2056 root *933u 0000 0,9 0 7351
>> anon_inode
>> java 2056 root *934u 0000 0,9 0 7351
>> anon_inode
>> java 2056 root *935u 0000 0,9 0 7351
>> anon_inode
>> java 2056 root *936u 0000 0,9 0 7351
>> anon_inode
>> java 2056 root *937u sock 0,7 0t0 73405 can't
>> identify protocol
>> java 2056 root *938u sock 0,7 0t0 73406 can't
>> identify protocol
>> java 2056 root *939u sock 0,7 0t0 73407 can't
>> identify protocol
>> java 2056 root *940u sock 0,7 0t0 71475 can't
>> identify protocol
>> java 2056 root *941u sock 0,7 0t0 74371 can't
>> identify protocol
>> java 2056 root *942u 0000 0,9 0 7351
>> anon_inode
>> java 2056 root *943u sock 0,7 0t0 74372 can't
>> identify protocol
>> java 2056 root *944u sock 0,7 0t0 74373 can't
>> identify protocol
>> java 2056 root *945u sock 0,7 0t0 74378 can't
>> identify protocol
>> java 2056 root *946u 0000 0,9 0 7351
>> anon_inode
>> java 2056 root *947u sock 0,7 0t0 71476 can't
>> identify protocol
>> java 2056 root *948u sock 0,7 0t0 74379 can't
>> identify protocol
>> java 2056 root *949u sock 0,7 0t0 76929 can't
>> identify protocol
>> java 2056 root *950u 0000 0,9 0 7351
>> anon_inode
>> java 2056 root *951u IPv6 76922 0t0 TCP
>> dripcast:https->c-67-185-79-130.hsd1.wa.comcast.net:61281 (ESTABLISHED)
>> java 2056 root *952u sock 0,7 0t0 76923 can't
>> identify protocol
>> java 2056 root *953u sock 0,7 0t0 76930 can't
>> identify protocol
>> java 2056 root *954u IPv6 76924 0t0 TCP
>> dripcast:https->outbound.kenshoo.com:14337 (ESTABLISHED)
>> java 2056 root *955u sock 0,7 0t0 76925 can't
>> identify protocol
>> java 2056 root *956u IPv6 76926 0t0 TCP
>> dripcast:https->kpersfw.state.ks.us:62427 (CLOSE_WAIT)
>> java 2056 root *957u sock 0,7 0t0 76931 can't
>> identify protocol
>> java 2056 root *958u 0000 0,9 0 7351
>> anon_inode
>> java 2056 root *959u 0000 0,9 0 7351
>> anon_inode
>> java 2056 root *960u 0000 0,9 0 7351
>> anon_inode
>> java 2056 root *961u 0000 0,9 0 7351
>> anon_inode
>> java 2056 root *962u 0000 0,9 0 7351
>> anon_inode
>>
>>
>> Prashant
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Tomcat opening thousands of files
Posted by David kerber <dc...@verizon.net>.
On 6/16/2014 12:57 PM, Prashant Deva wrote:
> I am using Tomcat 7.0.53 with Spring 4 websockets (spring's SockJS support).
> I notice that tomcat opens thousands of files.
Correction: your application opens thousands of files, or leaves them
open when it should have closed them.
> Just running for a few hours I noticed, it opened 16,552 files:
>
> root@dripcast:~# lsof -p 22691 | wc -l
> 16552
>
> Our logs are full of this exception thrown tons of times every second:
> org.apache.tomcat.util.net.AprEndpoint$Acceptor run
> SEVERE: Socket accept failed
> org.apache.tomcat.jni.Error: 24: Too many open files
> at org.apache.tomcat.jni.Socket.accept(Native Method)
> at
> org.apache.tomcat.util.net.AprEndpoint$Acceptor.run(AprEndpoint.java:992)
> at java.lang.Thread.run(Thread.java:745)
>
>
> Here is a sample of some of the open files, using lsof.
> What can we do to fix this?
>
> java 2056 root *904u sock 0,7 0t0 77846 can't
> identify protocol
> java 2056 root *905u IPv6 76695 0t0 TCP
> dripcast:https->pppoe.95-55-165-198.dynamic.avangarddsl.ru:1376
> (ESTABLISHED)
> java 2056 root *906u IPv6 74369 0t0 TCP
> dripcast:https->fibre.backbonetechnology.com:46415 (ESTABLISHED)
> java 2056 root *907u IPv6 76696 0t0 TCP
> dripcast:https->outbound.kenshoo.com:14292 (ESTABLISHED)
> java 2056 root *908u sock 0,7 0t0 76697 can't
> identify protocol
> java 2056 root *909u sock 0,7 0t0 76698 can't
> identify protocol
> java 2056 root *910u 0000 0,9 0 7351 anon_inode
> java 2056 root *911u 0000 0,9 0 7351 anon_inode
> java 2056 root *912u 0000 0,9 0 7351 anon_inode
> java 2056 root *913u 0000 0,9 0 7351 anon_inode
> java 2056 root *914u 0000 0,9 0 7351 anon_inode
> java 2056 root *915u sock 0,7 0t0 76699 can't
> identify protocol
> java 2056 root *916u sock 0,7 0t0 76700 can't
> identify protocol
> java 2056 root *917u sock 0,7 0t0 76701 can't
> identify protocol
> java 2056 root *918u sock 0,7 0t0 76702 can't
> identify protocol
> java 2056 root *919u sock 0,7 0t0 77845 can't
> identify protocol
> java 2056 root *920u sock 0,7 0t0 76705 can't
> identify protocol
> java 2056 root *921u IPv6 76906 0t0 TCP
> dripcast:https->c-67-185-79-130.hsd1.wa.comcast.net:61280 (ESTABLISHED)
> java 2056 root *922u 0000 0,9 0 7351 anon_inode
> java 2056 root *923u sock 0,7 0t0 77847 can't
> identify protocol
> java 2056 root *924u sock 0,7 0t0 75196 can't
> identify protocol
> java 2056 root *925u sock 0,7 0t0 76907 can't
> identify protocol
> java 2056 root *926u IPv6 76908 0t0 TCP
> dripcast:https->cdm-75-108-116-63-nixa.mid.suddenlink.net:56996
> (ESTABLISHED)
> java 2056 root *927u 0000 0,9 0 7351 anon_inode
> java 2056 root *928u IPv6 76909 0t0 TCP
> dripcast:https->fibre.backbonetechnology.com:46416 (ESTABLISHED)
> java 2056 root *929u sock 0,7 0t0 76910 can't
> identify protocol
> java 2056 root *930u sock 0,7 0t0 76911 can't
> identify protocol
> java 2056 root *931u IPv6 76912 0t0 TCP
> dripcast:https->outbound.kenshoo.com:14309 (ESTABLISHED)
> java 2056 root *932u 0000 0,9 0 7351 anon_inode
> java 2056 root *933u 0000 0,9 0 7351 anon_inode
> java 2056 root *934u 0000 0,9 0 7351 anon_inode
> java 2056 root *935u 0000 0,9 0 7351 anon_inode
> java 2056 root *936u 0000 0,9 0 7351 anon_inode
> java 2056 root *937u sock 0,7 0t0 73405 can't
> identify protocol
> java 2056 root *938u sock 0,7 0t0 73406 can't
> identify protocol
> java 2056 root *939u sock 0,7 0t0 73407 can't
> identify protocol
> java 2056 root *940u sock 0,7 0t0 71475 can't
> identify protocol
> java 2056 root *941u sock 0,7 0t0 74371 can't
> identify protocol
> java 2056 root *942u 0000 0,9 0 7351 anon_inode
> java 2056 root *943u sock 0,7 0t0 74372 can't
> identify protocol
> java 2056 root *944u sock 0,7 0t0 74373 can't
> identify protocol
> java 2056 root *945u sock 0,7 0t0 74378 can't
> identify protocol
> java 2056 root *946u 0000 0,9 0 7351 anon_inode
> java 2056 root *947u sock 0,7 0t0 71476 can't
> identify protocol
> java 2056 root *948u sock 0,7 0t0 74379 can't
> identify protocol
> java 2056 root *949u sock 0,7 0t0 76929 can't
> identify protocol
> java 2056 root *950u 0000 0,9 0 7351 anon_inode
> java 2056 root *951u IPv6 76922 0t0 TCP
> dripcast:https->c-67-185-79-130.hsd1.wa.comcast.net:61281 (ESTABLISHED)
> java 2056 root *952u sock 0,7 0t0 76923 can't
> identify protocol
> java 2056 root *953u sock 0,7 0t0 76930 can't
> identify protocol
> java 2056 root *954u IPv6 76924 0t0 TCP
> dripcast:https->outbound.kenshoo.com:14337 (ESTABLISHED)
> java 2056 root *955u sock 0,7 0t0 76925 can't
> identify protocol
> java 2056 root *956u IPv6 76926 0t0 TCP
> dripcast:https->kpersfw.state.ks.us:62427 (CLOSE_WAIT)
> java 2056 root *957u sock 0,7 0t0 76931 can't
> identify protocol
> java 2056 root *958u 0000 0,9 0 7351 anon_inode
> java 2056 root *959u 0000 0,9 0 7351 anon_inode
> java 2056 root *960u 0000 0,9 0 7351 anon_inode
> java 2056 root *961u 0000 0,9 0 7351 anon_inode
> java 2056 root *962u 0000 0,9 0 7351 anon_inode
>
>
> Prashant
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org