You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geronimo.apache.org by Neal Sanche <ne...@nsdev.org> on 2005/09/08 05:05:24 UTC

Longevity

Hi Guys,

I compiled up a new Geronimo, and then I left it running with the Server 
Info page displayed so I could see the cool AJX work there... and then I 
forgot about it for a day or so. When I got back it was saying:

Kernel
Kernel Boot Time     Tue Sep 06 11:35:32 MDT 2005
Kernel Up Time    
1 day 16 minutes 39 seconds

JVM
Java Version     1.5.0_03
Java Vendor     Sun Microsystems Inc.
Node     speedy/10.0.0.7
Current Memory Used    
63.49 MB
Most Memory Used    
63.56 MB
Total Memory Allocated    
63.56 MB
Available Processors     1

And my console was saying:

20:56:53,219 WARN  [ThreadedServer] EXCEPTION
java.lang.OutOfMemoryError: Java heap space

over and over again. If there's anything I can do to help find what's 
leaking let me know. I know I didn't really give Geronimo much room to 
grow, but when it started it was only taking 15MB of space, and I was 
impressed.

Cheers.

-Neal

Re: Longevity

Posted by Aaron Mulder <am...@alumni.princeton.edu>.
On Wed, 7 Sep 2005, Neal Sanche wrote:
> Well, I did some profiling, and found that the predominant object that 
> was being retained was:
> 
> org.apache.geronimo.kernel.basic.RawGetAttributeInvoker
> 
> There are also another slowly growing group of geronimo classes that
> seem to pile up, and they aren't getting garbage collected either:
> 
> org.apache.geronimo.kernel.basic.ProxyMethodInterceptor and many 
> internal classes of this class.

	Yeah, that means it's the proxies causing the problems.  Each
proxy has a ProxyMethodInterceptor and a bunch of RawGetAttributeInvokers.  
We need to figure out if there's some way to make the proxies self-dispose
if they become no longer referenced.  If not, then either the console code
needs to be *much* more disciplined or something needs to cache and reuse
proxies.

Aaron

> Neal Sanche wrote:
> 
> > Hi Aaron,
> >
> > Just in case it helps, I was connected to the SSL port of the web 
> > server when it happened. Out of Memory errors are not a lot of fun to 
> > debug, but maybe there's something I can try on my end to reproduce 
> > it, and figure out what objects are being retained. I'll let you know.
> >
> > -Neal
> >
> > Aaron Mulder wrote:
> >
> >>     It may be the console.  For example, the console creates a lot of
> >> proxies and doesn't close/release them -- I'm not sure whether that 
> >> would
> >> cause a memory leak, though I hope not.  I guess we need to try the test
> >> with some kind of JVM instrumentation going so we can see what's filling
> >> up the heap.
> >>
> >> Aaron
> >>
> >> On Wed, 7 Sep 2005, Neal Sanche wrote:
> >>
> >>  
> >>
> >>> Hi Guys,
> >>>
> >>> I compiled up a new Geronimo, and then I left it running with the 
> >>> Server Info page displayed so I could see the cool AJX work there... 
> >>> and then I forgot about it for a day or so. When I got back it was 
> >>> saying:
> >>>
> >>> Kernel
> >>> Kernel Boot Time     Tue Sep 06 11:35:32 MDT 2005
> >>> Kernel Up Time    1 day 16 minutes 39 seconds
> >>>
> >>> JVM
> >>> Java Version     1.5.0_03
> >>> Java Vendor     Sun Microsystems Inc.
> >>> Node     speedy/10.0.0.7
> >>> Current Memory Used    63.49 MB
> >>> Most Memory Used    63.56 MB
> >>> Total Memory Allocated    63.56 MB
> >>> Available Processors     1
> >>>
> >>> And my console was saying:
> >>>
> >>> 20:56:53,219 WARN  [ThreadedServer] EXCEPTION
> >>> java.lang.OutOfMemoryError: Java heap space
> >>>
> >>> over and over again. If there's anything I can do to help find 
> >>> what's leaking let me know. I know I didn't really give Geronimo 
> >>> much room to grow, but when it started it was only taking 15MB of 
> >>> space, and I was impressed.
> >>>
> >>> Cheers.
> >>>
> >>> -Neal
> >>>
> >>>   
> >>
> 
> 

Re: Longevity

Posted by Neal Sanche <ne...@nsdev.org>.
Hi again, Aaron,

Well, I did some profiling, and found that the predominant object that 
was being retained was:

org.apache.geronimo.kernel.basic.RawGetAttributeInvoker

Unfortunately I'm no profiling expert, and I may have done it wrong, but 
that's what seems to bubble to the top. When I run the garbage 
collector, some of them go away, but only a few (less than 5% on 
average). There are also another slowly growing group of geronimo 
classes that seem to pile up, and they aren't getting garbage collected 
either:

org.apache.geronimo.kernel.basic.ProxyMethodInterceptor and many 
internal classes of this class.

I'll leave it running overnight and see what happens. I do hope this 
information is helpful in some way.

Cheers.

-Neal

Neal Sanche wrote:

> Hi Aaron,
>
> Just in case it helps, I was connected to the SSL port of the web 
> server when it happened. Out of Memory errors are not a lot of fun to 
> debug, but maybe there's something I can try on my end to reproduce 
> it, and figure out what objects are being retained. I'll let you know.
>
> -Neal
>
> Aaron Mulder wrote:
>
>>     It may be the console.  For example, the console creates a lot of
>> proxies and doesn't close/release them -- I'm not sure whether that 
>> would
>> cause a memory leak, though I hope not.  I guess we need to try the test
>> with some kind of JVM instrumentation going so we can see what's filling
>> up the heap.
>>
>> Aaron
>>
>> On Wed, 7 Sep 2005, Neal Sanche wrote:
>>
>>  
>>
>>> Hi Guys,
>>>
>>> I compiled up a new Geronimo, and then I left it running with the 
>>> Server Info page displayed so I could see the cool AJX work there... 
>>> and then I forgot about it for a day or so. When I got back it was 
>>> saying:
>>>
>>> Kernel
>>> Kernel Boot Time     Tue Sep 06 11:35:32 MDT 2005
>>> Kernel Up Time    1 day 16 minutes 39 seconds
>>>
>>> JVM
>>> Java Version     1.5.0_03
>>> Java Vendor     Sun Microsystems Inc.
>>> Node     speedy/10.0.0.7
>>> Current Memory Used    63.49 MB
>>> Most Memory Used    63.56 MB
>>> Total Memory Allocated    63.56 MB
>>> Available Processors     1
>>>
>>> And my console was saying:
>>>
>>> 20:56:53,219 WARN  [ThreadedServer] EXCEPTION
>>> java.lang.OutOfMemoryError: Java heap space
>>>
>>> over and over again. If there's anything I can do to help find 
>>> what's leaking let me know. I know I didn't really give Geronimo 
>>> much room to grow, but when it started it was only taking 15MB of 
>>> space, and I was impressed.
>>>
>>> Cheers.
>>>
>>> -Neal
>>>
>>>   
>>


Re: Longevity

Posted by Neal Sanche <ne...@nsdev.org>.
Hi Aaron,

Just in case it helps, I was connected to the SSL port of the web server 
when it happened. Out of Memory errors are not a lot of fun to debug, 
but maybe there's something I can try on my end to reproduce it, and 
figure out what objects are being retained. I'll let you know.

-Neal

Aaron Mulder wrote:

>	It may be the console.  For example, the console creates a lot of
>proxies and doesn't close/release them -- I'm not sure whether that would
>cause a memory leak, though I hope not.  I guess we need to try the test
>with some kind of JVM instrumentation going so we can see what's filling
>up the heap.
>
>Aaron
>
>On Wed, 7 Sep 2005, Neal Sanche wrote:
>
>  
>
>>Hi Guys,
>>
>>I compiled up a new Geronimo, and then I left it running with the Server 
>>Info page displayed so I could see the cool AJX work there... and then I 
>>forgot about it for a day or so. When I got back it was saying:
>>
>>Kernel
>>Kernel Boot Time     Tue Sep 06 11:35:32 MDT 2005
>>Kernel Up Time    
>>1 day 16 minutes 39 seconds
>>
>>JVM
>>Java Version     1.5.0_03
>>Java Vendor     Sun Microsystems Inc.
>>Node     speedy/10.0.0.7
>>Current Memory Used    
>>63.49 MB
>>Most Memory Used    
>>63.56 MB
>>Total Memory Allocated    
>>63.56 MB
>>Available Processors     1
>>
>>And my console was saying:
>>
>>20:56:53,219 WARN  [ThreadedServer] EXCEPTION
>>java.lang.OutOfMemoryError: Java heap space
>>
>>over and over again. If there's anything I can do to help find what's 
>>leaking let me know. I know I didn't really give Geronimo much room to 
>>grow, but when it started it was only taking 15MB of space, and I was 
>>impressed.
>>
>>Cheers.
>>
>>-Neal
>>
>>    
>>


Re: Longevity

Posted by Aaron Mulder <am...@alumni.princeton.edu>.
	It may be the console.  For example, the console creates a lot of
proxies and doesn't close/release them -- I'm not sure whether that would
cause a memory leak, though I hope not.  I guess we need to try the test
with some kind of JVM instrumentation going so we can see what's filling
up the heap.

Aaron

On Wed, 7 Sep 2005, Neal Sanche wrote:

> Hi Guys,
> 
> I compiled up a new Geronimo, and then I left it running with the Server 
> Info page displayed so I could see the cool AJX work there... and then I 
> forgot about it for a day or so. When I got back it was saying:
> 
> Kernel
> Kernel Boot Time     Tue Sep 06 11:35:32 MDT 2005
> Kernel Up Time    
> 1 day 16 minutes 39 seconds
> 
> JVM
> Java Version     1.5.0_03
> Java Vendor     Sun Microsystems Inc.
> Node     speedy/10.0.0.7
> Current Memory Used    
> 63.49 MB
> Most Memory Used    
> 63.56 MB
> Total Memory Allocated    
> 63.56 MB
> Available Processors     1
> 
> And my console was saying:
> 
> 20:56:53,219 WARN  [ThreadedServer] EXCEPTION
> java.lang.OutOfMemoryError: Java heap space
> 
> over and over again. If there's anything I can do to help find what's 
> leaking let me know. I know I didn't really give Geronimo much room to 
> grow, but when it started it was only taking 15MB of space, and I was 
> impressed.
> 
> Cheers.
> 
> -Neal
>