Geronimo 1.1.1 and Liferay 4, memory problems on startup and hot deployment

[Paul ANDERSON <pa...@wta.dz>]

We have had trouble deploying Liferay 4.2.0 on Geronimo 1.1.1 out of the box with Liferay distro where they are prepacked together. I tried the 4.1.2 pro WAR file too and had the same memory problems.
We immediately get PermGen memory use errors on Java 1.5.09 on RHEL 3, 4 and Windows XP even with more than 1 gig allocated. Sometimes it happens on startup, otherwise on the first hot deployment. Restarting after every deployment helps if the deployment didn't crash the server. On Tomcat 5.5, LR runs fine with little memory.

The July 2006 archives contain discussion about blocking 1.1.1 because of this problem but it seems not to have happened.

The user list contains a recent question, Liferay 4.2.0 and Geronimo 1.1.1 about the same problem on startup.

And on JIRA there is issue GERONIMO-2644 that may be related:-
"Fix leaking ClassLoaders
Created: 11/Dec/06 12:04 PM   Updated: 11/Dec/06 12:19 PM 
Component/s:
kernel 
Affects Version/s:
1.2 , 2.0-M1 
Fix Version/s:
1.2 , 2.0-M1 
"Looks like we're leaking ClassLoaders. If you deploy/undeploy an app the ClassLoader(s) created for the app are not being GC'ed. Eventually, you'll run out of PermGen space. I assume that this is also causing the PermGen problems when running tests."

Will this problem be fixed in 1.2 and if so, when is 1.2 release planned?
We wanted to adopt Geronimo and go into production with it for running Liferay, but this is a killer for us.
Thanks
Paul.

Re: Geronimo 1.1.1 and Liferay 4, memory problems on startup and hot deployment

[Kevan Miller <ke...@gmail.com>]

On Dec 20, 2006, at 8:46 AM, Jeff Genender wrote:

> Kevan M,
>
> Do you have any thoughts on the leaking classloaders?

Hey Jeff,
AFAIK, Geronimo 1.1.1 running on JRE 1.4.2 was clean. I ran Daytrader  
through multiple deploy/undeploy cycles. Daytrader is great, in this  
regard, because it hits a lot of our builder code... There are some  
temporal deploy/undeploy leaks, where a Thread Local variable will  
keep a classloader alive until the Thread is re-used by the system.  
This could cause issues when undeploying/deploying very large apps  
with lots of classes.

I am seeing Classloaders being leaked in Geronimo 1.2-beta and 2.0- 
M1. It's looking like Java 1.5 is behaving differently than 1.4. So,  
it's possible that Geronimo 1.1.1 running on 1.5 has similar  
problems. I've been trying to get some time to look at these issues,  
but just have tenative data, at the moment...

If PermGen problems are being encountered with a single app  
deployment, then you'll definitely need to bump up MaxPermSize, as  
you've suggested.

FYI, The following option will cause the VM to print GC statistics  
(including PermGen info).

-XX:+PrintGCDetails

>
> I have run Liferay 4.1.2 and 4.2 on Geronimo 1.1.1 setting the -Xms1G
> -Xmx1G and have never run into any memory problems what-so-ever, with
> the memory used at about 110M after it's up and running.  I have  
> run it
> on the Mac.
>
> Paul,
>
> Have you tried upping the permgen memory parameter on boot?  Try  
> adding
> this to JAVA_OPTS:
>
> -XX:MaxPermSize=128m
>
> Jeff
>
>
> Paul ANDERSON wrote:
>> We have had trouble deploying Liferay 4.2.0 on Geronimo 1.1.1 out  
>> of the box with Liferay distro where they are prepacked together.  
>> I tried the 4.1.2 pro WAR file too and had the same memory problems.
>> We immediately get PermGen memory use errors on Java 1.5.09 on  
>> RHEL 3, 4 and Windows XP even with more than 1 gig allocated.  
>> Sometimes it happens on startup, otherwise on the first hot  
>> deployment. Restarting after every deployment helps if the  
>> deployment didn't crash the server. On Tomcat 5.5, LR runs fine  
>> with little memory.
>>
>> The July 2006 archives contain discussion about blocking 1.1.1  
>> because of this problem but it seems not to have happened.

IIRC, all known memory-related issues got cleaned up for 1.1.1.  
However, I didn't run any tests on Java 5. So, it's possible that  
there may be some Java 5 issues for G 1.1.1, also...

>>
>> The user list contains a recent question, Liferay 4.2.0 and  
>> Geronimo 1.1.1 about the same problem on startup.

I hadn't noticed this. Thanks for the pointer Paul.

>>
>> And on JIRA there is issue GERONIMO-2644 that may be related:-
>> "Fix leaking ClassLoaders
>> Created: 11/Dec/06 12:04 PM   Updated: 11/Dec/06 12:19 PM
>> Component/s:
>> kernel
>> Affects Version/s:
>> 1.2 , 2.0-M1
>> Fix Version/s:
>> 1.2 , 2.0-M1
>> "Looks like we're leaking ClassLoaders. If you deploy/undeploy an  
>> app the ClassLoader(s) created for the app are not being GC'ed.  
>> Eventually, you'll run out of PermGen space. I assume that this is  
>> also causing the PermGen problems when running tests."
>>
>> Will this problem be fixed in 1.2 and if so, when is 1.2 release  
>> planned?

Well, IMO, this is a must-fix for a release (but just fine to have  
this problem in a beta/milestone).

I don't know what kind of target Dain has for releasing 1.2. I  
certainly hope soon...

>> We wanted to adopt Geronimo and go into production with it for  
>> running Liferay, but this is a killer for us.

Paul,
Well, let's get this problem cleaned up, then... ;-) I may ping you  
later with questions about how you're deploying Liferay, but will run  
some other tests, first...

If you could gather some additional data using PrintGCDetails and  
open a Jira, that would be great. However, I should be able to make  
progress in the meantime...

--kevan

Re: Geronimo 1.1.1 and Liferay 4, memory problems on startup and hot deployment

[Jeff Genender <jg...@apache.org>]

Kevan M,

Do you have any thoughts on the leaking classloaders?

I have run Liferay 4.1.2 and 4.2 on Geronimo 1.1.1 setting the -Xms1G
-Xmx1G and have never run into any memory problems what-so-ever, with
the memory used at about 110M after it's up and running.  I have run it
on the Mac.

Paul,

Have you tried upping the permgen memory parameter on boot?  Try adding
this to JAVA_OPTS:

-XX:MaxPermSize=128m

Jeff


Paul ANDERSON wrote:
> We have had trouble deploying Liferay 4.2.0 on Geronimo 1.1.1 out of the box with Liferay distro where they are prepacked together. I tried the 4.1.2 pro WAR file too and had the same memory problems.
> We immediately get PermGen memory use errors on Java 1.5.09 on RHEL 3, 4 and Windows XP even with more than 1 gig allocated. Sometimes it happens on startup, otherwise on the first hot deployment. Restarting after every deployment helps if the deployment didn't crash the server. On Tomcat 5.5, LR runs fine with little memory.
> 
> The July 2006 archives contain discussion about blocking 1.1.1 because of this problem but it seems not to have happened.
> 
> The user list contains a recent question, Liferay 4.2.0 and Geronimo 1.1.1 about the same problem on startup.
> 
> And on JIRA there is issue GERONIMO-2644 that may be related:-
> "Fix leaking ClassLoaders
> Created: 11/Dec/06 12:04 PM   Updated: 11/Dec/06 12:19 PM 
> Component/s:
> kernel 
> Affects Version/s:
> 1.2 , 2.0-M1 
> Fix Version/s:
> 1.2 , 2.0-M1 
> "Looks like we're leaking ClassLoaders. If you deploy/undeploy an app the ClassLoader(s) created for the app are not being GC'ed. Eventually, you'll run out of PermGen space. I assume that this is also causing the PermGen problems when running tests."
> 
> Will this problem be fixed in 1.2 and if so, when is 1.2 release planned?
> We wanted to adopt Geronimo and go into production with it for running Liferay, but this is a killer for us.
> Thanks
> Paul.