You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@roller.apache.org by Anil Gangolli <an...@busybuddha.org> on 2006/01/02 16:30:18 UTC
Re: JVM Fails with Too Many Files Open
If you have the lsof utility on the system (most linuxes), try using
that for diagnosis.
As root run: lsof -p <the_tomcat_pid>.
Glenn Nielsen wrote:
>I have had a third failure. Once again after 9-10 days. There is
>definitley some sort of file descriptor leak but I couldn't find
>any more information. Neither netstat or fstat showed anything
>unusual. Neither did a full thread stack dump of the JVM.
>
>Regards,
>
>Glenn
>
>On Thu, Dec 22, 2005 at 11:56:08PM -0500, Max Rudman wrote:
>
>
>>We've definitely seen this problem of Tomcat running out of sockets.
>>I our case it's caused by Lucene which seems to leak file descriptors
>>to search index. Unix shows lots of "orphaned" file descriptors to
>>various segment files inside the index.
>>
>>Max
>>
>>
>>On Dec 22, 2005, at 11:45 PM, Anil Gangolli wrote:
>>
>>
>>
>>>I've filed ROL-946, which identifies one probable source of a
>>>connection leak. I'll fix this when I return in a week or so if it
>>>hasn't already been addressed.
>>>
>>>--a
>>>
>>>Anil Gangolli wrote:
>>>
>>>
>>>
>>>>Yeah. Any idea of where the leaks are will help. Please do file
>>>>a bug because I'd like to have a thread specific to this there.
>>>>
>>>>Glenn Nielsen wrote:
>>>>
>>>>
>>>>
>>>>>Next time it happens I will do some forensics before I restart
>>>>>Tomcat.
>>>>>
>>>>>Things such as a JVM full thread stack dump, netstat, fstat, etc.
>>>>>
>>>>>Then I may have a better idea where the file descriptor leak is.
>>>>>
>>>>>One thing I have found is that the way GC works may impact the
>>>>>number of file descriptors. If you don't explicitly close some
>>>>>IO streams the file descriptor associated with that stream will
>>>>>still exist until the object is finalized during garbage collection.
>>>>>
>>>>>Regards,
>>>>>
>>>>>Glenn Nielsen
>>>>>
>>>>>On Sun, Dec 18, 2005 at 08:59:22AM -0800, Allen Gilliland wrote:
>>>>>
>>>>>
>>>>>
>>>>>>Actually, you may also want to check the search stuff. i have
>>>>>>seen that
>>>>>>same thing happen with Lucene in other apps before.
>>>>>>
>>>>>>-- Allen
>>>>>>
>>>>>>
>>>>>>On Sun, 2005-12-18 at 06:38, Anil Gangolli wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>>Sounds like something we need to track and look into. Please
>>>>>>>file a bug report on our issue tracker: http://
>>>>>>>opensource2.atlassian.com/projects/roller/secure/Dashboard.jspa
>>>>>>>
>>>>>>>--a.
>>>>>>>
>>>>>>>Glenn Nielsen wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>Since upgrading to roller 1.2 and configuring several feeds in
>>>>>>>>the Planet Tomcat has failed twice due to running out of file
>>>>>>>>descriptors. Both times the failure occurred about 10 days
>>>>>>>>after Tomcat was started. I suspect the background threads
>>>>>>>>for the Planet may be leaking file descriptors for sockets
>>>>>>>>but haven't confirmed this yet.
>>>>>>>>
>>>>>>>>FreeBSD 5.3, Java 1.4, Tomcat 4.1, Roller 1.2.
>>>>>>>>
>>>>>>>>Has anyone had similar problems?
>>>>>>>>
>>>>>>>>Regards,
>>>>>>>>
>>>>>>>>Glenn Nielsen
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>Glenn
>
>
>
>
Re: JVM Fails with Too Many Files Open
Posted by Glenn Nielsen <gl...@earthdome.org>.
I am using FreeBSD and just installed the lsof port.
Thanks,
Glenn
On Mon, Jan 02, 2006 at 07:30:18AM -0800, Anil Gangolli wrote:
> If you have the lsof utility on the system (most linuxes), try using
> that for diagnosis.
> As root run: lsof -p <the_tomcat_pid>.
>
> Glenn Nielsen wrote:
>
> >I have had a third failure. Once again after 9-10 days. There is
> >definitley some sort of file descriptor leak but I couldn't find
> >any more information. Neither netstat or fstat showed anything
> >unusual. Neither did a full thread stack dump of the JVM.
> >
> >Regards,
> >
> >Glenn
> >
> >On Thu, Dec 22, 2005 at 11:56:08PM -0500, Max Rudman wrote:
> >
> >
> >>We've definitely seen this problem of Tomcat running out of sockets.
> >>I our case it's caused by Lucene which seems to leak file descriptors
> >>to search index. Unix shows lots of "orphaned" file descriptors to
> >>various segment files inside the index.
> >>
> >>Max
> >>
> >>
> >>On Dec 22, 2005, at 11:45 PM, Anil Gangolli wrote:
> >>
> >>
> >>
> >>>I've filed ROL-946, which identifies one probable source of a
> >>>connection leak. I'll fix this when I return in a week or so if it
> >>>hasn't already been addressed.
> >>>
> >>>--a
> >>>
> >>>Anil Gangolli wrote:
> >>>
> >>>
> >>>
> >>>>Yeah. Any idea of where the leaks are will help. Please do file
> >>>>a bug because I'd like to have a thread specific to this there.
> >>>>
> >>>>Glenn Nielsen wrote:
> >>>>
> >>>>
> >>>>
> >>>>>Next time it happens I will do some forensics before I restart
> >>>>>Tomcat.
> >>>>>
> >>>>>Things such as a JVM full thread stack dump, netstat, fstat, etc.
> >>>>>
> >>>>>Then I may have a better idea where the file descriptor leak is.
> >>>>>
> >>>>>One thing I have found is that the way GC works may impact the
> >>>>>number of file descriptors. If you don't explicitly close some
> >>>>>IO streams the file descriptor associated with that stream will
> >>>>>still exist until the object is finalized during garbage collection.
> >>>>>
> >>>>>Regards,
> >>>>>
> >>>>>Glenn Nielsen
> >>>>>
> >>>>>On Sun, Dec 18, 2005 at 08:59:22AM -0800, Allen Gilliland wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>>Actually, you may also want to check the search stuff. i have
> >>>>>>seen that
> >>>>>>same thing happen with Lucene in other apps before.
> >>>>>>
> >>>>>>-- Allen
> >>>>>>
> >>>>>>
> >>>>>>On Sun, 2005-12-18 at 06:38, Anil Gangolli wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>Sounds like something we need to track and look into. Please
> >>>>>>>file a bug report on our issue tracker: http://
> >>>>>>>opensource2.atlassian.com/projects/roller/secure/Dashboard.jspa
> >>>>>>>
> >>>>>>>--a.
> >>>>>>>
> >>>>>>>Glenn Nielsen wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>Since upgrading to roller 1.2 and configuring several feeds in
> >>>>>>>>the Planet Tomcat has failed twice due to running out of file
> >>>>>>>>descriptors. Both times the failure occurred about 10 days
> >>>>>>>>after Tomcat was started. I suspect the background threads
> >>>>>>>>for the Planet may be leaking file descriptors for sockets
> >>>>>>>>but haven't confirmed this yet.
> >>>>>>>>
> >>>>>>>>FreeBSD 5.3, Java 1.4, Tomcat 4.1, Roller 1.2.
> >>>>>>>>
> >>>>>>>>Has anyone had similar problems?
> >>>>>>>>
> >>>>>>>>Regards,
> >>>>>>>>
> >>>>>>>>Glenn Nielsen
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >Glenn
> >
> >
> >
> >
>
Glenn