memory use & performance

[Mike French <Mi...@screwfix.com>]

I have some comments and suggestions about memory use (leaks)
and performance in version 1.5.3-1. 
The main focus of my attention was <antcall> tasks.

I see about 2Mb allocated per antcall, independent of any actual 
work performed. I traced this to through the <antcall> target 
(CallTarget class) creating an <ant> task (Ant class),
which builds a large project structure, containing many
cross-references between Project, Tasks and Targets.
The main culprits are:
  - Project keeping a list of createdTasks
    (I just commented this out as a test!)
  - Ant task internal project keeping properties 
    and other tasks containing references back to the project

I see some attempt in Ant::execute to free up the memory
in a finally clause:

  Enumeration enum = properties.elements();
  while (enum.hasMoreElements()) {
    Property p = (Property) enum.nextElement();
    p.setProject(null);
  }

I found that freeing other references helped too:

  properties.clear();
  references.clear();

  enum = newProject.getTargets().elements();
  while (enum.hasMoreElements()) {
    Target t = (Target) enum.nextElement();
    t.setProject(null);
  }
  newProject.getTargets().clear();

Similar back linked structures seem to be created
for every target invocation, e.g. Calltarget::execute.
I would suggest a 'clear()' method on all these objects
to free up all held references, which could be used to 
help the garbage collector break circular dependencies,
and perhaps use of WeakHashMap in place of Hashtables.

I also noticed performance for n-target build.xml
degrading as n^2, which soon gets a little out of control.
I again traced this through the <antcall> target 
(CallTarget class) creating an <ant> task (Ant class) 
which in turn calls ProjectHelper.configureProject,
which re-parses the whole of the build.xml and
recreates a huge new project structure every time !
It seems unfortunate that the whole build.xml has to be 
replayed for each and every local <antcall> invocation.
Again the underlying issue is that <antcall> uses <ant>,
I suggest antcall should use the enclosing project by reference.

Apologies if all these issues have been fixed in 1.6 !
I haven't had a chance to migrate yet.

Mike



This email has been checked for all viruses at the internet level by MailDefender. MailDefender is powered by Sophos anti-virus (www.maildefender.net)

Re: memory use & performance

[Peter Reilly <pe...@corvil.com>]

You need to test on 1.6.0.

Most of the problems have not been fixed, but
things are more complicated.

The need for <antcall> has been reduced due
to the addition of <import> and <macrodef>.
(in 1.7 <local> or similar will be added).

Removing the references to tasks is difficult,
for example the following horrible build script relies on it:

  <target name="sub">
    <echo id="theEcho"/>
  </target>

  <target name="sub1">
    <script language="javascript"><![CDATA[
      theEcho.setMessage("In sub1");
      sub.execute();
    ]]></script>
  </target>

  <target name="sub2">
    <script language="javascript"><![CDATA[
      theEcho.setMessage("In sub2");
      sub.execute();
    ]]></script>
  </target>


refernces to tasks are also keep in RuntimeConfigurable objects, and these
are used during reflection.

The best fix would be to stop keeping refernces to tasks when not needed and
then the individual tasks would not need to have a "memory cleanup" 
finally clause in
the execute method (which can cause bugs - see bug 26310).
This is java and not c++! >:o

Peter

Mike French wrote:

>I have some comments and suggestions about memory use (leaks)
>and performance in version 1.5.3-1. 
>The main focus of my attention was <antcall> tasks.
>
>I see about 2Mb allocated per antcall, independent of any actual 
>work performed. I traced this to through the <antcall> target 
>(CallTarget class) creating an <ant> task (Ant class),
>which builds a large project structure, containing many
>cross-references between Project, Tasks and Targets.
>The main culprits are:
>  - Project keeping a list of createdTasks
>    (I just commented this out as a test!)
>  - Ant task internal project keeping properties 
>    and other tasks containing references back to the project
>
>I see some attempt in Ant::execute to free up the memory
>in a finally clause:
>
>  Enumeration enum = properties.elements();
>  while (enum.hasMoreElements()) {
>    Property p = (Property) enum.nextElement();
>    p.setProject(null);
>  }
>
>I found that freeing other references helped too:
>
>  properties.clear();
>  references.clear();
>
>  enum = newProject.getTargets().elements();
>  while (enum.hasMoreElements()) {
>    Target t = (Target) enum.nextElement();
>    t.setProject(null);
>  }
>  newProject.getTargets().clear();
>
>Similar back linked structures seem to be created
>for every target invocation, e.g. Calltarget::execute.
>I would suggest a 'clear()' method on all these objects
>to free up all held references, which could be used to 
>help the garbage collector break circular dependencies,
>and perhaps use of WeakHashMap in place of Hashtables.
>
>I also noticed performance for n-target build.xml
>degrading as n^2, which soon gets a little out of control.
>I again traced this through the <antcall> target 
>(CallTarget class) creating an <ant> task (Ant class) 
>which in turn calls ProjectHelper.configureProject,
>which re-parses the whole of the build.xml and
>recreates a huge new project structure every time !
>It seems unfortunate that the whole build.xml has to be 
>replayed for each and every local <antcall> invocation.
>Again the underlying issue is that <antcall> uses <ant>,
>I suggest antcall should use the enclosing project by reference.
>
>Apologies if all these issues have been fixed in 1.6 !
>I haven't had a chance to migrate yet.
>
>Mike
>
>
>
>This email has been checked for all viruses at the internet level by MailDefender. MailDefender is powered by Sophos anti-virus (www.maildefender.net)
>
>  
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org