subant holds onto classloaders

[Troy Self <ts...@bbn.com>]

I am using the ivy:buildlist / subant combination to operate the build for a
large project. The project consists of about 20 different sub-projects. The
sub-projects each depend on a combination of other sub-projects as well as
some 3rd party libraries. Ivy takes care of building them in order for me.
That's all great.

The issue I have is that subant does all of this in a single process. As a
result, as each sub-project resolves its compile dependencies and then later
resolves its junit dependencies, the primary JVM process is retaining all of
the classloaders and never unloading them. The result is that my build
eventually fails due to running out of memory in the PermGen space.

Does anyone know of a way that I can get the sub-projects to build in the
correct order, but still have ant fork a new process for each one? This way
the primary JVM would not keep adding and retaining classloaders.

Btw, when running manually, I can get around the issue by increasing the
PermGen space using -XX:MaxPermSize=256m. However, my continuous build
system (Cruise Control) seems to want to run ant with the default JVM
parameters and this solution only delays the issue anyway.

Thanks,

-- tBs

Re: subant holds onto classloaders

[Maarten Coene <ma...@yahoo.com>]

This is more a question for ant-users mailing list since it is not Ivy that is 
locking the jars, but rather it will be the javac or junit tasks.

Maarten




----- Original Message ----
From: Mandie Smith <ma...@houseofgnomes.net>
To: ivy-user@ant.apache.org
Sent: Fri, September 10, 2010 8:29:03 PM
Subject: Re: subant holds onto classloaders

We actually use ivy:buildlist/exec instead of subant to avoid that exact
scenario.  This also allows you to pass in custom JVM args to the
subprocesses.  We use ant-contrib to do a for loop through the sorted build
list.

Here's a condensed example:

<ivy:buildlist reference="build-path" haltonerror="true"
onMissingDescriptor="fail">
  <fileset refid="buildFiles.fileset" />
</ivy:buildlist>

<for list="${toString:build-path}" param="moduleBuildFile" delimiter=":">
  <sequential>
    <exec executable="ant" dir="${projects.dir}/${module.name}"
failonerror="false"
output="${projects.dir}/${module.name}/log/${module.name}.log"
outputproperty="exec.output" resultproperty="exec.result">
      <arg line="${module.buildproperties}" />
      <arg line="${module.targets}" />
    </exec>
  </sequential>
</for>


Mandie

On Wed, Sep 8, 2010 at 8:44 AM, Troy Self <ts...@bbn.com> wrote:

> I am using the ivy:buildlist / subant combination to operate the build for
> a
> large project. The project consists of about 20 different sub-projects. The
> sub-projects each depend on a combination of other sub-projects as well as
> some 3rd party libraries. Ivy takes care of building them in order for me.
> That's all great.
>
> The issue I have is that subant does all of this in a single process. As a
> result, as each sub-project resolves its compile dependencies and then
> later
> resolves its junit dependencies, the primary JVM process is retaining all
> of
> the classloaders and never unloading them. The result is that my build
> eventually fails due to running out of memory in the PermGen space.
>
> Does anyone know of a way that I can get the sub-projects to build in the
> correct order, but still have ant fork a new process for each one? This way
> the primary JVM would not keep adding and retaining classloaders.
>
> Btw, when running manually, I can get around the issue by increasing the
> PermGen space using -XX:MaxPermSize=256m. However, my continuous build
> system (Cruise Control) seems to want to run ant with the default JVM
> parameters and this solution only delays the issue anyway.
>
> Thanks,
>
> -- tBs
>
>



      

Re: subant holds onto classloaders

[Mandie Smith <ma...@houseofgnomes.net>]

We actually use ivy:buildlist/exec instead of subant to avoid that exact
scenario.  This also allows you to pass in custom JVM args to the
subprocesses.  We use ant-contrib to do a for loop through the sorted build
list.

Here's a condensed example:

<ivy:buildlist reference="build-path" haltonerror="true"
onMissingDescriptor="fail">
  <fileset refid="buildFiles.fileset" />
</ivy:buildlist>

<for list="${toString:build-path}" param="moduleBuildFile" delimiter=":">
  <sequential>
    <exec executable="ant" dir="${projects.dir}/${module.name}"
failonerror="false"
output="${projects.dir}/${module.name}/log/${module.name}.log"
outputproperty="exec.output" resultproperty="exec.result">
      <arg line="${module.buildproperties}" />
      <arg line="${module.targets}" />
    </exec>
  </sequential>
</for>


Mandie

On Wed, Sep 8, 2010 at 8:44 AM, Troy Self <ts...@bbn.com> wrote:

> I am using the ivy:buildlist / subant combination to operate the build for
> a
> large project. The project consists of about 20 different sub-projects. The
> sub-projects each depend on a combination of other sub-projects as well as
> some 3rd party libraries. Ivy takes care of building them in order for me.
> That's all great.
>
> The issue I have is that subant does all of this in a single process. As a
> result, as each sub-project resolves its compile dependencies and then
> later
> resolves its junit dependencies, the primary JVM process is retaining all
> of
> the classloaders and never unloading them. The result is that my build
> eventually fails due to running out of memory in the PermGen space.
>
> Does anyone know of a way that I can get the sub-projects to build in the
> correct order, but still have ant fork a new process for each one? This way
> the primary JVM would not keep adding and retaining classloaders.
>
> Btw, when running manually, I can get around the issue by increasing the
> PermGen space using -XX:MaxPermSize=256m. However, my continuous build
> system (Cruise Control) seems to want to run ant with the default JVM
> parameters and this solution only delays the issue anyway.
>
> Thanks,
>
> -- tBs
>
>