You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Hilco Wijbenga <hi...@gmail.com> on 2012/05/12 00:56:06 UTC
Memory Leak?
Hi all,
I have noticed that in a fairly large reactor build the build dies
close to the end (about 10 modules to go out of around 90). It simply
runs out of memory. Restarting it with -rf allows it to finish without
issues. There is a limit to how much memory I can give Maven... :-)
So apparently, Maven does not release memory before continuing on with
the next module? Or would this be something that is specific to a
particular plugin (in my case, it's undoubtedly the GWT Maven plugin)?
Cheers,
Hilco
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Memory Leak?
Posted by Kristian Rosenvold <kr...@gmail.com>.
You *will* want to make sure you're running the latest versions of
most plugins, since a few leaks of this type have been fixed over the
last year or so. surefire had one.
Kristian
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Memory Leak?
Posted by Jörg Schaible <jo...@gmx.de>.
Hi Oliver,
Olivier Lamy wrote:
> No issue with multi threads build ? see
> https://jira.codehaus.org/browse/MCOMPILER-170
> I have to say depending on os/jdk it may cause weird issue. I will try
> to release a new version soon with the fix.
We're back to M2 because of https://jira.codehaus.org/browse/MNG-5207, so we
cannot use multi-threaded builds anyway. :-)
- Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Memory Leak?
Posted by Olivier Lamy <ol...@apache.org>.
No issue with multi threads build ? see
https://jira.codehaus.org/browse/MCOMPILER-170
I have to say depending on os/jdk it may cause weird issue. I will try
to release a new version soon with the fix.
2012/5/12 Jörg Schaible <jo...@gmx.de>:
> Hi Hilco,
>
> Hilco Wijbenga wrote:
>
>> Hi all,
>>
>> I have noticed that in a fairly large reactor build the build dies
>> close to the end (about 10 modules to go out of around 90). It simply
>> runs out of memory. Restarting it with -rf allows it to finish without
>> issues. There is a limit to how much memory I can give Maven... :-)
>>
>> So apparently, Maven does not release memory before continuing on with
>> the next module? Or would this be something that is specific to a
>> particular plugin (in my case, it's undoubtedly the GWT Maven plugin)?
>
> did you already try the new compiler plugin 2.4? It includes now
> https://jira.codehaus.org/browse/PLXCOMP-202. We already upgraded the new
> plexus-compiler dependency manually before and it had a dramatic (positive)
> effect on build times and memory consumption (our complete build has ~500
> artifacts).
>
> Cheers,
> Jörg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
--
Olivier Lamy
Talend: http://coders.talend.com
http://twitter.com/olamy | http://linkedin.com/in/olamy
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Memory Leak?
Posted by Jörg Schaible <jo...@gmx.de>.
Hi Hilco,
Hilco Wijbenga wrote:
> Hi all,
>
> I have noticed that in a fairly large reactor build the build dies
> close to the end (about 10 modules to go out of around 90). It simply
> runs out of memory. Restarting it with -rf allows it to finish without
> issues. There is a limit to how much memory I can give Maven... :-)
>
> So apparently, Maven does not release memory before continuing on with
> the next module? Or would this be something that is specific to a
> particular plugin (in my case, it's undoubtedly the GWT Maven plugin)?
did you already try the new compiler plugin 2.4? It includes now
https://jira.codehaus.org/browse/PLXCOMP-202. We already upgraded the new
plexus-compiler dependency manually before and it had a dramatic (positive)
effect on build times and memory consumption (our complete build has ~500
artifacts).
Cheers,
Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Memory Leak?
Posted by Olivier Lamy <ol...@apache.org>.
2012/5/12 Hilco Wijbenga <hi...@gmail.com>:
> Hi all,
>
> I have noticed that in a fairly large reactor build the build dies
> close to the end (about 10 modules to go out of around 90). It simply
> runs out of memory. Restarting it with -rf allows it to finish without
> issues. There is a limit to how much memory I can give Maven... :-)
>
> So apparently, Maven does not release memory before continuing on with
> the next module? Or would this be something that is specific to a
> particular plugin (in my case, it's undoubtedly the GWT Maven plugin)?
AFAIK gwt maven plugin fork a process so must not have any impact on
the maven memory.
Any stack trace ?
Are you using -T for multi thread builds ?
>
> Cheers,
> Hilco
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
--
Olivier Lamy
Talend: http://coders.talend.com
http://twitter.com/olamy | http://linkedin.com/in/olamy
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Memory Leak?
Posted by Stuart McCulloch <mc...@gmail.com>.
On 12 May 2012, at 12:05, Mark Struberg wrote:
> It could also be a proxy problem of course. How is guice doing proxies?
Maven currently uses guice-no_aop http://code.google.com/p/google-guice/wiki/OptionalAOP so any proxies would use the standard JVM proxy mechanism.
That said, I wouldn't expect many proxies to be generated with Maven - they're typically only used when there are circular references between constructors.
> Proxies are also loaded via the ClassLoader and consume perm gen space.
>
> LieGrue,
> strub
>
> ----- Original Message -----
>> From: Stuart McCulloch <mc...@gmail.com>
>> To: Maven Users List <us...@maven.apache.org>
>> Cc:
>> Sent: Saturday, May 12, 2012 11:36 AM
>> Subject: Re: Memory Leak?
>>
>> On 12 May 2012, at 07:32, Hilco Wijbenga wrote:
>>
>>> On 11 May 2012 22:47, Anders Hammar <an...@hammar.net> wrote:
>>>> This is not uncommon for large multi-module builds. You need to
>>>> increase the memory available for Maven, such as the heap depending on
>>>> the error you're getting.
>>>> Do this by setting the MAVEN_OPTS env variable.
>>>
>>> I know, it is already getting 2GB. I can't set it any higher (Maven
>>> refuses to even start then).
>>
>> You might be running out of PermGen space - this is a separate heap in the
>> HotSpot JVM used for classes, etc. Normally the default PermGen limit is ok, but
>> can become an issue when running large apps or doing lots of code generation /
>> compilation. The other thing is that bumping the main heap size doesn't
>> change the default PermGen limit - instead you need to use something like:
>>
>> -XX:MaxPermSize=256m
>>
>> https://cwiki.apache.org/MAVEN/outofmemoryerror.html
>>
>>>> My experience is that this is mainly due to the plugins being used in
>>>> the build, not Maven core. Are you using Maven 3? Maven 3 core has a
>>>> smaller memory footprint than Maven 2.
>>>
>>> Maven 3.0.3. Should Maven not simply release plugins (and the memory
>>> they use) between module builds?
>>
>> IIRC the Maven plugin manager should release mojos after each execution - of
>> course this depends on the plugin not having a memory leak involving static
>> members, which can keep memory around until the classloader is unloaded (which
>> can take longer depending on other references). Also if you're hitting the
>> PermGen limit then it's more about the amount of classes and interned
>> strings you're loading rather the amount of objects being created.
>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>>> For additional commands, e-mail: users-help@maven.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Memory Leak?
Posted by Mark Struberg <st...@yahoo.de>.
It could also be a proxy problem of course. How is guice doing proxies?
Proxies are also loaded via the ClassLoader and consume perm gen space.
LieGrue,
strub
----- Original Message -----
> From: Stuart McCulloch <mc...@gmail.com>
> To: Maven Users List <us...@maven.apache.org>
> Cc:
> Sent: Saturday, May 12, 2012 11:36 AM
> Subject: Re: Memory Leak?
>
> On 12 May 2012, at 07:32, Hilco Wijbenga wrote:
>
>> On 11 May 2012 22:47, Anders Hammar <an...@hammar.net> wrote:
>>> This is not uncommon for large multi-module builds. You need to
>>> increase the memory available for Maven, such as the heap depending on
>>> the error you're getting.
>>> Do this by setting the MAVEN_OPTS env variable.
>>
>> I know, it is already getting 2GB. I can't set it any higher (Maven
>> refuses to even start then).
>
> You might be running out of PermGen space - this is a separate heap in the
> HotSpot JVM used for classes, etc. Normally the default PermGen limit is ok, but
> can become an issue when running large apps or doing lots of code generation /
> compilation. The other thing is that bumping the main heap size doesn't
> change the default PermGen limit - instead you need to use something like:
>
> -XX:MaxPermSize=256m
>
> https://cwiki.apache.org/MAVEN/outofmemoryerror.html
>
>>> My experience is that this is mainly due to the plugins being used in
>>> the build, not Maven core. Are you using Maven 3? Maven 3 core has a
>>> smaller memory footprint than Maven 2.
>>
>> Maven 3.0.3. Should Maven not simply release plugins (and the memory
>> they use) between module builds?
>
> IIRC the Maven plugin manager should release mojos after each execution - of
> course this depends on the plugin not having a memory leak involving static
> members, which can keep memory around until the classloader is unloaded (which
> can take longer depending on other references). Also if you're hitting the
> PermGen limit then it's more about the amount of classes and interned
> strings you're loading rather the amount of objects being created.
>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Memory Leak?
Posted by Stuart McCulloch <mc...@gmail.com>.
On 12 May 2012, at 07:32, Hilco Wijbenga wrote:
> On 11 May 2012 22:47, Anders Hammar <an...@hammar.net> wrote:
>> This is not uncommon for large multi-module builds. You need to
>> increase the memory available for Maven, such as the heap depending on
>> the error you're getting.
>> Do this by setting the MAVEN_OPTS env variable.
>
> I know, it is already getting 2GB. I can't set it any higher (Maven
> refuses to even start then).
You might be running out of PermGen space - this is a separate heap in the HotSpot JVM used for classes, etc. Normally the default PermGen limit is ok, but can become an issue when running large apps or doing lots of code generation / compilation. The other thing is that bumping the main heap size doesn't change the default PermGen limit - instead you need to use something like:
-XX:MaxPermSize=256m
https://cwiki.apache.org/MAVEN/outofmemoryerror.html
>> My experience is that this is mainly due to the plugins being used in
>> the build, not Maven core. Are you using Maven 3? Maven 3 core has a
>> smaller memory footprint than Maven 2.
>
> Maven 3.0.3. Should Maven not simply release plugins (and the memory
> they use) between module builds?
IIRC the Maven plugin manager should release mojos after each execution - of course this depends on the plugin not having a memory leak involving static members, which can keep memory around until the classloader is unloaded (which can take longer depending on other references). Also if you're hitting the PermGen limit then it's more about the amount of classes and interned strings you're loading rather the amount of objects being created.
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Memory Leak?
Posted by Hilco Wijbenga <hi...@gmail.com>.
On 11 May 2012 22:47, Anders Hammar <an...@hammar.net> wrote:
> This is not uncommon for large multi-module builds. You need to
> increase the memory available for Maven, such as the heap depending on
> the error you're getting.
> Do this by setting the MAVEN_OPTS env variable.
I know, it is already getting 2GB. I can't set it any higher (Maven
refuses to even start then).
> My experience is that this is mainly due to the plugins being used in
> the build, not Maven core. Are you using Maven 3? Maven 3 core has a
> smaller memory footprint than Maven 2.
Maven 3.0.3. Should Maven not simply release plugins (and the memory
they use) between module builds?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Memory Leak?
Posted by Anders Hammar <an...@hammar.net>.
This is not uncommon for large multi-module builds. You need to
increase the memory available for Maven, such as the heap depending on
the error you're getting.
Do this by setting the MAVEN_OPTS env variable.
My experience is that this is mainly due to the plugins being used in
the build, not Maven core. Are you using Maven 3? Maven 3 core has a
smaller memory footprint than Maven 2.
/Anders
On Sat, May 12, 2012 at 12:56 AM, Hilco Wijbenga
<hi...@gmail.com> wrote:
> Hi all,
>
> I have noticed that in a fairly large reactor build the build dies
> close to the end (about 10 modules to go out of around 90). It simply
> runs out of memory. Restarting it with -rf allows it to finish without
> issues. There is a limit to how much memory I can give Maven... :-)
>
> So apparently, Maven does not release memory before continuing on with
> the next module? Or would this be something that is specific to a
> particular plugin (in my case, it's undoubtedly the GWT Maven plugin)?
>
> Cheers,
> Hilco
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: [Solved] Re: Memory Leak?
Posted by Olivier Lamy <ol...@apache.org>.
2012/5/14 Hilco Wijbenga <hi...@gmail.com>:
> On 11 May 2012 15:56, Hilco Wijbenga <hi...@gmail.com> wrote:
>> I have noticed that in a fairly large reactor build the build dies
>> close to the end (about 10 modules to go out of around 90). It simply
>> runs out of memory. Restarting it with -rf allows it to finish without
>> issues. There is a limit to how much memory I can give Maven... :-)
>>
>> So apparently, Maven does not release memory before continuing on with
>> the next module? Or would this be something that is specific to a
>> particular plugin (in my case, it's undoubtedly the GWT Maven plugin)?
>
> I changed a number of things so I'm not sure exactly which one did the
> trick ... but I think we can make a good guess. :-) Oh, and this is
> all with Maven 3.0.3.
Cool and 3.0.4 ?
>
> I finally upgraded my system from 32bit to 64bit (GNU/Linux). I also
> went from the Sun JDK (1.6.0.32) to IcedTea (7.2.1). Finally, I
> upgraded the Maven Compiler plugin to 2.4. I believe it is mainly the
> first two changes that fixed things. Strangely enough, I can now run
> the full build with only 1GB (and I could probably use even less). I
> had 2GB before and that was still not enough...
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
--
Olivier Lamy
Talend: http://coders.talend.com
http://twitter.com/olamy | http://linkedin.com/in/olamy
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: [Solved] Re: Memory Leak?
Posted by Jörg Schaible <Jo...@scalaris.com>.
Hilco Wijbenga wrote:
> On 14 May 2012 23:46, Jörg Schaible <Jo...@scalaris.com> wrote:
>> Hilco Wijbenga wrote:
>>
>>> On 11 May 2012 15:56, Hilco Wijbenga <hi...@gmail.com> wrote:
>>>> I have noticed that in a fairly large reactor build the build dies
>>>> close to the end (about 10 modules to go out of around 90). It simply
>>>> runs out of memory. Restarting it with -rf allows it to finish without
>>>> issues. There is a limit to how much memory I can give Maven... :-)
>>>>
>>>> So apparently, Maven does not release memory before continuing on with
>>>> the next module? Or would this be something that is specific to a
>>>> particular plugin (in my case, it's undoubtedly the GWT Maven plugin)?
>>>
>>> I changed a number of things so I'm not sure exactly which one did the
>>> trick ... but I think we can make a good guess. :-) Oh, and this is
>>> all with Maven 3.0.3.
>>>
>>> I finally upgraded my system from 32bit to 64bit (GNU/Linux). I also
>>> went from the Sun JDK (1.6.0.32) to IcedTea (7.2.1). Finally, I
>>> upgraded the Maven Compiler plugin to 2.4. I believe it is mainly the
>>> first two changes that fixed things.
>>
>> I'd bet against it.
>
> Count yourself lucky that you never got the chance because you would
> have lost that bet. :-)
>
>>> Strangely enough, I can now run
>>> the full build with only 1GB (and I could probably use even less). I
>>> had 2GB before and that was still not enough...
>>
>> Use an older compiler plugin and your memory problem is back ;-)
>
> After your challenge, I decided to go back to 2.3.2 and actually try
> it. :-) The build still runs fine: my memory problem did not come
> back. So while I am sure 2.4 is an improvement over 2.3.2, it does not
> seem to have anything to do with my memory problems.
Oh, well. Good to know anyway. Thanks for trying :-)
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: [Solved] Re: Memory Leak?
Posted by Hilco Wijbenga <hi...@gmail.com>.
On 14 May 2012 23:46, Jörg Schaible <Jo...@scalaris.com> wrote:
> Hilco Wijbenga wrote:
>
>> On 11 May 2012 15:56, Hilco Wijbenga <hi...@gmail.com> wrote:
>>> I have noticed that in a fairly large reactor build the build dies
>>> close to the end (about 10 modules to go out of around 90). It simply
>>> runs out of memory. Restarting it with -rf allows it to finish without
>>> issues. There is a limit to how much memory I can give Maven... :-)
>>>
>>> So apparently, Maven does not release memory before continuing on with
>>> the next module? Or would this be something that is specific to a
>>> particular plugin (in my case, it's undoubtedly the GWT Maven plugin)?
>>
>> I changed a number of things so I'm not sure exactly which one did the
>> trick ... but I think we can make a good guess. :-) Oh, and this is
>> all with Maven 3.0.3.
>>
>> I finally upgraded my system from 32bit to 64bit (GNU/Linux). I also
>> went from the Sun JDK (1.6.0.32) to IcedTea (7.2.1). Finally, I
>> upgraded the Maven Compiler plugin to 2.4. I believe it is mainly the
>> first two changes that fixed things.
>
> I'd bet against it.
Count yourself lucky that you never got the chance because you would
have lost that bet. :-)
>> Strangely enough, I can now run
>> the full build with only 1GB (and I could probably use even less). I
>> had 2GB before and that was still not enough...
>
> Use an older compiler plugin and your memory problem is back ;-)
After your challenge, I decided to go back to 2.3.2 and actually try
it. :-) The build still runs fine: my memory problem did not come
back. So while I am sure 2.4 is an improvement over 2.3.2, it does not
seem to have anything to do with my memory problems.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: [Solved] Re: Memory Leak?
Posted by Jörg Schaible <Jo...@scalaris.com>.
Hilco Wijbenga wrote:
> On 11 May 2012 15:56, Hilco Wijbenga <hi...@gmail.com> wrote:
>> I have noticed that in a fairly large reactor build the build dies
>> close to the end (about 10 modules to go out of around 90). It simply
>> runs out of memory. Restarting it with -rf allows it to finish without
>> issues. There is a limit to how much memory I can give Maven... :-)
>>
>> So apparently, Maven does not release memory before continuing on with
>> the next module? Or would this be something that is specific to a
>> particular plugin (in my case, it's undoubtedly the GWT Maven plugin)?
>
> I changed a number of things so I'm not sure exactly which one did the
> trick ... but I think we can make a good guess. :-) Oh, and this is
> all with Maven 3.0.3.
>
> I finally upgraded my system from 32bit to 64bit (GNU/Linux). I also
> went from the Sun JDK (1.6.0.32) to IcedTea (7.2.1). Finally, I
> upgraded the Maven Compiler plugin to 2.4. I believe it is mainly the
> first two changes that fixed things.
I'd bet against it.
> Strangely enough, I can now run
> the full build with only 1GB (and I could probably use even less). I
> had 2GB before and that was still not enough...
Use an older compiler plugin and your memory problem is back ;-)
Cheers,
Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
[Solved] Re: Memory Leak?
Posted by Hilco Wijbenga <hi...@gmail.com>.
On 11 May 2012 15:56, Hilco Wijbenga <hi...@gmail.com> wrote:
> I have noticed that in a fairly large reactor build the build dies
> close to the end (about 10 modules to go out of around 90). It simply
> runs out of memory. Restarting it with -rf allows it to finish without
> issues. There is a limit to how much memory I can give Maven... :-)
>
> So apparently, Maven does not release memory before continuing on with
> the next module? Or would this be something that is specific to a
> particular plugin (in my case, it's undoubtedly the GWT Maven plugin)?
I changed a number of things so I'm not sure exactly which one did the
trick ... but I think we can make a good guess. :-) Oh, and this is
all with Maven 3.0.3.
I finally upgraded my system from 32bit to 64bit (GNU/Linux). I also
went from the Sun JDK (1.6.0.32) to IcedTea (7.2.1). Finally, I
upgraded the Maven Compiler plugin to 2.4. I believe it is mainly the
first two changes that fixed things. Strangely enough, I can now run
the full build with only 1GB (and I could probably use even less). I
had 2GB before and that was still not enough...
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org