You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ivy-user@ant.apache.org by jgunz <sl...@twcny.rr.com> on 2007/08/10 21:12:39 UTC
Subant calls utilizing Ivy leak memory
I'm getting "java.lang.OutOfMemoryError: PermGen space" errors when using
subant calls to iterate over a series of sub-projects which utilize Ivy. It
seems each call has to re-parse the ivysettings.xml file and continually
eats up memory.
I've mocked up a simple example project that illustrates the problem. Ignore
the Groovy usage; I just did that to loop over the same sub-project multiple
times to simulate the effect of having numerous sub-projects.
I recommend reducing Ant's maximum memory size so that you see the problem
faster by setting ANT_OPTS=-XX:MaxPermSize=32m
The following project should run out of the box and generate an
OutOfMemoryError.
http://www.nabble.com/file/p12097331/ivySubantFailure.zip
ivySubantFailure.zip
--
View this message in context: http://www.nabble.com/Subant-calls-utilizing-Ivy-leak-memory-tf4250538.html#a12097331
Sent from the ivy-user mailing list archive at Nabble.com.
RE: Subant calls utilizing Ivy leak memory
Posted by Gilles Scokart <gs...@gmail.com>.
Two possible approaches are:
- Use the loaderRef argument.
- Use some properties to skip the typedef when not already done.
Gilles
> -----Original Message-----
> From: jgunz [mailto:slobby@twcny.rr.com]
> Sent: lundi 13 août 2007 15:11
> To: ivy-user@incubator.apache.org
> Subject: Re: Subant calls utilizing Ivy leak memory
>
>
> I've had this problem for awhile and have done various things to minimize
> the
> impact on my project (eg. increased MaxPermSize) but the problem is still
> there and resurfaces every so often as we add more memory intensive tasks.
>
> I believe the core of the problem is not the taskdef but the reconfiguring
> of Ivy. I haven't had a chance to dig into the code and I'm completely
> unfamiliar with Ant/Ivy's memory management. If you look at my example
> project and set the maxpermsize to 32mb, it fails very quickly. If you
> then
> simply comment out the <ivy:resolve/> call in the subproject, the failure
> does not occur.
>
> Gilles,
> You mentioned you made sure all subants were using the same classloader
> and
> configured Ivy only once -- how did you ensure this?
>
>
>
> Gilles Scokart wrote:
> >
> > I also had similar problem.
> >
> > In my case the origin was that I reloaded the ivy task every time. I
> > fixed the problem by making sure that all subants where reusing the
> > same class loader and that I configured ivy only once.
> >
> > Gilles
> >
> > 2007/8/11, Jeffrey Blattman <je...@gmail.com>:
> >> we had the same problem. however, my conclusion was that using Ivy was
> >> not the root cause but rather the straw that broke the camel's back.
> the
> >> research i did showed the using the "ant" and "antcall" targets are
> >> memory hungry and are at the root cause. i assume when you say "iterate
> >> over a series of subprojects" you are using the "ant" task.
> >>
> >> you can work around this by setting the MaxPermSize to some larger
> >> value. you can set this in your environment by doing:
> >>
> >> $ export ANT_OPTS="$ANT_OPTS -Xmx... -XX:MaxPermSize=..."
> >>
> >> where ... are some values you'll need to play with.
> >>
> >>
> >> jgunz wrote:
> >> > I'm getting "java.lang.OutOfMemoryError: PermGen space" errors when
> >> using
> >> > subant calls to iterate over a series of sub-projects which utilize
> >> Ivy. It
> >> > seems each call has to re-parse the ivysettings.xml file and
> >> continually
> >> > eats up memory.
> >> >
> >> > I've mocked up a simple example project that illustrates the problem.
> >> Ignore
> >> > the Groovy usage; I just did that to loop over the same sub-project
> >> multiple
> >> > times to simulate the effect of having numerous sub-projects.
> >> >
> >> > I recommend reducing Ant's maximum memory size so that you see the
> >> problem
> >> > faster by setting ANT_OPTS=-XX:MaxPermSize=32m
> >> >
> >> > The following project should run out of the box and generate an
> >> > OutOfMemoryError.
> >> > http://www.nabble.com/file/p12097331/ivySubantFailure.zip
> >> > ivySubantFailure.zip
> >> >
> >> >
> >>
> >
> >
> > --
> > Gilles SCOKART
> >
> >
>
> --
> View this message in context: http://www.nabble.com/Subant-calls-
> utilizing-Ivy-leak-memory-tf4250538.html#a12125931
> Sent from the ivy-user mailing list archive at Nabble.com.
Re: Subant calls utilizing Ivy leak memory
Posted by jgunz <sl...@twcny.rr.com>.
I've had this problem for awhile and have done various things to minimize the
impact on my project (eg. increased MaxPermSize) but the problem is still
there and resurfaces every so often as we add more memory intensive tasks.
I believe the core of the problem is not the taskdef but the reconfiguring
of Ivy. I haven't had a chance to dig into the code and I'm completely
unfamiliar with Ant/Ivy's memory management. If you look at my example
project and set the maxpermsize to 32mb, it fails very quickly. If you then
simply comment out the <ivy:resolve/> call in the subproject, the failure
does not occur.
Gilles,
You mentioned you made sure all subants were using the same classloader and
configured Ivy only once -- how did you ensure this?
Gilles Scokart wrote:
>
> I also had similar problem.
>
> In my case the origin was that I reloaded the ivy task every time. I
> fixed the problem by making sure that all subants where reusing the
> same class loader and that I configured ivy only once.
>
> Gilles
>
> 2007/8/11, Jeffrey Blattman <je...@gmail.com>:
>> we had the same problem. however, my conclusion was that using Ivy was
>> not the root cause but rather the straw that broke the camel's back. the
>> research i did showed the using the "ant" and "antcall" targets are
>> memory hungry and are at the root cause. i assume when you say "iterate
>> over a series of subprojects" you are using the "ant" task.
>>
>> you can work around this by setting the MaxPermSize to some larger
>> value. you can set this in your environment by doing:
>>
>> $ export ANT_OPTS="$ANT_OPTS -Xmx... -XX:MaxPermSize=..."
>>
>> where ... are some values you'll need to play with.
>>
>>
>> jgunz wrote:
>> > I'm getting "java.lang.OutOfMemoryError: PermGen space" errors when
>> using
>> > subant calls to iterate over a series of sub-projects which utilize
>> Ivy. It
>> > seems each call has to re-parse the ivysettings.xml file and
>> continually
>> > eats up memory.
>> >
>> > I've mocked up a simple example project that illustrates the problem.
>> Ignore
>> > the Groovy usage; I just did that to loop over the same sub-project
>> multiple
>> > times to simulate the effect of having numerous sub-projects.
>> >
>> > I recommend reducing Ant's maximum memory size so that you see the
>> problem
>> > faster by setting ANT_OPTS=-XX:MaxPermSize=32m
>> >
>> > The following project should run out of the box and generate an
>> > OutOfMemoryError.
>> > http://www.nabble.com/file/p12097331/ivySubantFailure.zip
>> > ivySubantFailure.zip
>> >
>> >
>>
>
>
> --
> Gilles SCOKART
>
>
--
View this message in context: http://www.nabble.com/Subant-calls-utilizing-Ivy-leak-memory-tf4250538.html#a12125931
Sent from the ivy-user mailing list archive at Nabble.com.
Re: Subant calls utilizing Ivy leak memory
Posted by Gilles Scokart <gs...@gmail.com>.
I also had similar problem.
In my case the origin was that I reloaded the ivy task every time. I
fixed the problem by making sure that all subants where reusing the
same class loader and that I configured ivy only once.
Gilles
2007/8/11, Jeffrey Blattman <je...@gmail.com>:
> we had the same problem. however, my conclusion was that using Ivy was
> not the root cause but rather the straw that broke the camel's back. the
> research i did showed the using the "ant" and "antcall" targets are
> memory hungry and are at the root cause. i assume when you say "iterate
> over a series of subprojects" you are using the "ant" task.
>
> you can work around this by setting the MaxPermSize to some larger
> value. you can set this in your environment by doing:
>
> $ export ANT_OPTS="$ANT_OPTS -Xmx... -XX:MaxPermSize=..."
>
> where ... are some values you'll need to play with.
>
>
> jgunz wrote:
> > I'm getting "java.lang.OutOfMemoryError: PermGen space" errors when using
> > subant calls to iterate over a series of sub-projects which utilize Ivy. It
> > seems each call has to re-parse the ivysettings.xml file and continually
> > eats up memory.
> >
> > I've mocked up a simple example project that illustrates the problem. Ignore
> > the Groovy usage; I just did that to loop over the same sub-project multiple
> > times to simulate the effect of having numerous sub-projects.
> >
> > I recommend reducing Ant's maximum memory size so that you see the problem
> > faster by setting ANT_OPTS=-XX:MaxPermSize=32m
> >
> > The following project should run out of the box and generate an
> > OutOfMemoryError.
> > http://www.nabble.com/file/p12097331/ivySubantFailure.zip
> > ivySubantFailure.zip
> >
> >
>
--
Gilles SCOKART
Re: Subant calls utilizing Ivy leak memory
Posted by Steve Loughran <st...@apache.org>.
Jeffrey Blattman wrote:
> we had the same problem. however, my conclusion was that using Ivy was
> not the root cause but rather the straw that broke the camel's back. the
> research i did showed the using the "ant" and "antcall" targets are
> memory hungry and are at the root cause. i assume when you say "iterate
> over a series of subprojects" you are using the "ant" task.
>
> you can work around this by setting the MaxPermSize to some larger
> value. you can set this in your environment by doing:
>
> $ export ANT_OPTS="$ANT_OPTS -Xmx... -XX:MaxPermSize=..."
>
> where ... are some values you'll need to play with.
We've discussed this -its a mixture of Ant and Ivy failing to work together
my problems: http://jira.smartfrog.org/jira/browse/SFOS-167
as raised with Ant http://issues.apache.org/bugzilla/show_bug.cgi?id=42742
Perm gen space is where task defs are being stored, for some reason they
are not being released when a sub build finishes. We modified our
<typdef> calls of ivy to be skipped if ivy was already defined (using
the <typefound> condition); this stopped the memory leaks.
One thing nobody is sure of is whether this is an Ant bug or a Java
problem. If we could identify it as being ant-related, and put together
a patch, I'd commit it.
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action http://antbook.org/
Re: Subant calls utilizing Ivy leak memory
Posted by Jeffrey Blattman <je...@gmail.com>.
we had the same problem. however, my conclusion was that using Ivy was
not the root cause but rather the straw that broke the camel's back. the
research i did showed the using the "ant" and "antcall" targets are
memory hungry and are at the root cause. i assume when you say "iterate
over a series of subprojects" you are using the "ant" task.
you can work around this by setting the MaxPermSize to some larger
value. you can set this in your environment by doing:
$ export ANT_OPTS="$ANT_OPTS -Xmx... -XX:MaxPermSize=..."
where ... are some values you'll need to play with.
jgunz wrote:
> I'm getting "java.lang.OutOfMemoryError: PermGen space" errors when using
> subant calls to iterate over a series of sub-projects which utilize Ivy. It
> seems each call has to re-parse the ivysettings.xml file and continually
> eats up memory.
>
> I've mocked up a simple example project that illustrates the problem. Ignore
> the Groovy usage; I just did that to loop over the same sub-project multiple
> times to simulate the effect of having numerous sub-projects.
>
> I recommend reducing Ant's maximum memory size so that you see the problem
> faster by setting ANT_OPTS=-XX:MaxPermSize=32m
>
> The following project should run out of the box and generate an
> OutOfMemoryError.
> http://www.nabble.com/file/p12097331/ivySubantFailure.zip
> ivySubantFailure.zip
>
>