You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ant.apache.org by Stefan Bodewig <bo...@apache.org> on 2004/01/15 16:19:58 UTC
New Launcher and JPackage RPMs
Hi,
today I found myself discussing with the JPackage folks on their list
about the problems the RPM setup has with the new Launcher.
They put all the installed jars into a single directory and let config
files put together the classpath before they invoke Ant IIUC.
So far we've told people that the new Launcher would pull in all jars
from ANT_HOME/lib (and in addition from ~/.ant/lib, but we didn't say
that very loud) - but this is not true. It pulls in all jars from the
directory containing ant-launcher.jar, no matter where it is.
In stock installs this is the same, but not in the RPM installation
where this behavior pulls in the whole Java universe.
I'm not convinced that we need to adjust anything, but it is an
interesting thing to note.
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: New Launcher and JPackage RPMs
Posted by Steve Cohen <sc...@javactivity.org>.
Please ignore. This last comment by me was added in a reply to the wrong
message.
On Friday 16 January 2004 11:18 am, Steve Cohen wrote:
> Well, we at commons-net would have been rushing the release to meet ant's
> "deadline" and there are internal refactorings which we would have liked to
> have included that we were planning to postpone in the rush, but now will
> take the time to get right, so I doubt that commons-net-1.2.0 will be done
> in time for ant 1.6.1 and therefore we shouldn't update the ant manual just
> yet.
>
> I think the best procedure will be for commons-net to work at it's own best
> pace and when it's done, notify the ant list, via bugzilla and via posts to
> the ant-dev and ant-user lists. At that point, any ant users needing this
> functionality would be able to grab the commons-net-1.2.0 jar and get the
> basic functionality. At that point also, coding could begin on the HEAD
> branch of ant and the documentation in time for 1.7 (or maybe 1.6.2 if that
> comes to pass).
>
> On Friday 16 January 2004 9:31 am, Antoine Lévy-Lambert wrote:
> > Peter Reilly wrote:
> > > Antoine Lévy-Lambert wrote:
> > >> I am +1 to get this into ant 1.6.1.
> > >
> > > (in relation to static map of jarfile->manifest class path in
> > > AntClassLoader2).
> > >
> > > Ok I will commit that.
> > >
> > > Another optimization I tried was a quick hack to DefBase to have a
> > > static field containing the default classloader, so it
> > > gets set once. This did speed up the typedef the second and
> > > subsequent times and reduced the
> > > time for the test to 1.6 second (from 3 and thus below the 1.5.4 times
> > > (2 second) when using the crimson xml parser).
> >
> > Sounds useful.
> >
> > > However it is a complete hack, and does not deal with non-default
> > > classpaths like:
> > > <typedef classpath="${antlib.jar}"
> > > resource="net/sf/antcontrib/antcontrib.properties"/>
> >
> > Do you mean : your change is bringing an optimization for taskdefs which
> > are done based on jars which have already been loaded by the launcher,
> > presumably because they are in $ANT_HOME/lib or in a -lib directory ?
> > And not if the taskdef is requiring extra jars or directories which were
> > not included when ant got started ?
> >
> > Cheers,
> >
> > Antoine
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> > For additional commands, e-mail: dev-help@ant.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> For additional commands, e-mail: dev-help@ant.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: New Launcher and JPackage RPMs
Posted by Steve Cohen <sc...@javactivity.org>.
Well, we at commons-net would have been rushing the release to meet ant's
"deadline" and there are internal refactorings which we would have liked to
have included that we were planning to postpone in the rush, but now will
take the time to get right, so I doubt that commons-net-1.2.0 will be done in
time for ant 1.6.1 and therefore we shouldn't update the ant manual just yet.
I think the best procedure will be for commons-net to work at it's own best
pace and when it's done, notify the ant list, via bugzilla and via posts to
the ant-dev and ant-user lists. At that point, any ant users needing this
functionality would be able to grab the commons-net-1.2.0 jar and get the
basic functionality. At that point also, coding could begin on the HEAD
branch of ant and the documentation in time for 1.7 (or maybe 1.6.2 if that
comes to pass).
On Friday 16 January 2004 9:31 am, Antoine Lévy-Lambert wrote:
> Peter Reilly wrote:
> > Antoine Lévy-Lambert wrote:
> >> I am +1 to get this into ant 1.6.1.
> >
> > (in relation to static map of jarfile->manifest class path in
> > AntClassLoader2).
> >
> > Ok I will commit that.
> >
> > Another optimization I tried was a quick hack to DefBase to have a
> > static field containing the default classloader, so it
> > gets set once. This did speed up the typedef the second and
> > subsequent times and reduced the
> > time for the test to 1.6 second (from 3 and thus below the 1.5.4 times
> > (2 second) when using the crimson xml parser).
>
> Sounds useful.
>
> > However it is a complete hack, and does not deal with non-default
> > classpaths like:
> > <typedef classpath="${antlib.jar}"
> > resource="net/sf/antcontrib/antcontrib.properties"/>
>
> Do you mean : your change is bringing an optimization for taskdefs which
> are done based on jars which have already been loaded by the launcher,
> presumably because they are in $ANT_HOME/lib or in a -lib directory ?
> And not if the taskdef is requiring extra jars or directories which were
> not included when ant got started ?
>
> Cheers,
>
> Antoine
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> For additional commands, e-mail: dev-help@ant.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: New Launcher and JPackage RPMs
Posted by Peter Reilly <pe...@corvil.com>.
Antoine Lévy-Lambert wrote:
> Peter Reilly wrote:
>
>> Antoine Lévy-Lambert wrote:
>>
>>> I am +1 to get this into ant 1.6.1.
>>
>>
>>
>> (in relation to static map of jarfile->manifest class path in
>> AntClassLoader2).
>>
>> Ok I will commit that.
>>
>> Another optimization I tried was a quick hack to DefBase to have a
>> static field containing the default classloader, so it
>> gets set once. This did speed up the typedef the second and
>> subsequent times and reduced the
>> time for the test to 1.6 second (from 3 and thus below the 1.5.4
>> times (2 second) when using the crimson xml parser).
>
>
>
> Sounds useful.
>
>> However it is a complete hack, and does not deal with non-default
>> classpaths like:
>> <typedef classpath="${antlib.jar}"
>> resource="net/sf/antcontrib/antcontrib.properties"/>
>>
> Do you mean : your change is bringing an optimization for taskdefs
> which are done based on jars which have already been loaded by the
> launcher, presumably because they are in $ANT_HOME/lib or in a -lib
> directory ?
No, the optimization is that if the <typedef/> does not define a
classpath, a cached classloader is used. This
cached classloader gets set once (the first call).
Jose's more general caching solution souds more promising.
Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: New Launcher and JPackage RPMs
Posted by Antoine Lévy-Lambert <an...@antbuild.com>.
Peter Reilly wrote:
> Antoine Lévy-Lambert wrote:
>
>> I am +1 to get this into ant 1.6.1.
>
>
> (in relation to static map of jarfile->manifest class path in
> AntClassLoader2).
>
> Ok I will commit that.
>
> Another optimization I tried was a quick hack to DefBase to have a
> static field containing the default classloader, so it
> gets set once. This did speed up the typedef the second and
> subsequent times and reduced the
> time for the test to 1.6 second (from 3 and thus below the 1.5.4 times
> (2 second) when using the crimson xml parser).
Sounds useful.
> However it is a complete hack, and does not deal with non-default
> classpaths like:
> <typedef classpath="${antlib.jar}"
> resource="net/sf/antcontrib/antcontrib.properties"/>
>
Do you mean : your change is bringing an optimization for taskdefs which
are done based on jars which have already been loaded by the launcher,
presumably because they are in $ANT_HOME/lib or in a -lib directory ?
And not if the taskdef is requiring extra jars or directories which were
not included when ant got started ?
Cheers,
Antoine
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: New Launcher and JPackage RPMs
Posted by Peter Reilly <pe...@corvil.com>.
Antoine L�vy-Lambert wrote:
> I am +1 to get this into ant 1.6.1.
(in relation to static map of jarfile->manifest class path in
AntClassLoader2).
Ok I will commit that.
Another optimization I tried was a quick hack to DefBase to have a
static field containing the default classloader, so it
gets set once. This did speed up the typedef the second and subsequent
times and reduced the
time for the test to 1.6 second (from 3 and thus below the 1.5.4 times
(2 second) when using the crimson xml parser).
However it is a complete hack, and does not deal with non-default
classpaths like:
<typedef classpath="${antlib.jar}"
resource="net/sf/antcontrib/antcontrib.properties"/>
Note that the problem is not so bad for the antlib: xml ns shortcut.
<project xmlns:ac="antlib:net.sf.antcontrib">
<target name="compile-mains">
<ac:foreach list="a,b,c,d,e,f,g,h,i" param="program"
target="compile"/>
</target>
<target name="compile">
..
</target>
</project>
This will in ant 1.6.1 cause only one typedef of the ant-contrib tasks.
(There
is a bug in 1.6.0 which causes the tasks to be loaded for each ant-call,
this
is fixed in cvs).
Peter
Re: New Launcher and JPackage RPMs
Posted by Antoine Lévy-Lambert <an...@antbuild.com>.
I am +1 to get this into ant 1.6.1.
Cheers,
Antoine
Peter Reilly wrote:
> Antoine Lévy-Lambert wrote:
>
>> Hi Peter,
>>
>> can we improve the new classloader so that the work of inspecting the
>> jars does not happen each time an <antcall/> is done ?
>
>
> Yes by keeping a static map of file->manifest class path.
>
> With the following:
> <project default="call-all">
> <taskdef resource="net/sf/antcontrib/antcontrib.properties" />
> <taskdef resource="cpptasks.tasks"/>
> <typedef resource="cpptasks.types"/>
> <target name="call-all">
> <foreach list="1,2,3,4,5,6,7,8,9,10" param="name" target="doit"/>
> <echo>Done</echo>
> </target>
> <target name="doit">
> <foreach list="1,2,3,4,5,6,7,8,9,10" param="name" target="doit2"/>
> </target>
> <target name="doit2"/>
> </project>
>
> and the included patch, the time is reduced from 6 seconds to 3
> seconds - not
> a great improvement considering that there is 100 ant-calls, but it is
> a improvement!
>
> Peter
>
>------------------------------------------------------------------------
>
>Index: src/main/org/apache/tools/ant/loader/AntClassLoader2.java
>===================================================================
>RCS file: /home/cvs/ant/src/main/org/apache/tools/ant/loader/AntClassLoader2.java,v
>retrieving revision 1.7
>diff -u -r1.7 AntClassLoader2.java
>--- src/main/org/apache/tools/ant/loader/AntClassLoader2.java 17 Jul 2003 10:36:27 -0000 1.7
>+++ src/main/org/apache/tools/ant/loader/AntClassLoader2.java 15 Jan 2004 17:34:19 -0000
>@@ -1,7 +1,7 @@
> /*
> * The Apache Software License, Version 1.1
> *
>- * Copyright (c) 2003 The Apache Software Foundation. All rights
>+ * Copyright (c) 2003-2004 The Apache Software Foundation. All rights
> * reserved.
> *
> * Redistribution and use in source and binary forms, with or without
>@@ -68,6 +68,9 @@
> import java.net.URL;
> import java.net.MalformedURLException;
> import java.util.zip.ZipEntry;
>+import java.util.Collections;
>+import java.util.HashMap;
>+import java.util.Map;
> import java.util.StringTokenizer;
> import org.apache.tools.ant.util.FileUtils;
>
>@@ -81,6 +84,9 @@
> /** Instance of a utility class to use for file operations. */
> private FileUtils fileUtils;
>
>+ /** Static map of jar file/time to manifiest class-path entries */
>+ private static Map pathMap = Collections.synchronizedMap(new HashMap());
>+
> /**
> * Constructor
> */
>@@ -272,36 +278,46 @@
> return;
> }
>
>+ String absPathPlusTimeAndLength =
>+ pathComponent.getAbsolutePath() + pathComponent.lastModified() + "-"
>+ + pathComponent.length();
> String classpath = null;
>- ZipFile jarFile = null;
>- InputStream manifestStream = null;
>- try {
>- jarFile = new ZipFile(pathComponent);
>- manifestStream
>- = jarFile.getInputStream(new ZipEntry("META-INF/MANIFEST.MF"));
>-
>- if (manifestStream == null) {
>- return;
>- }
>- Reader manifestReader
>- = new InputStreamReader(manifestStream, "UTF-8");
>- org.apache.tools.ant.taskdefs.Manifest manifest
>- = new org.apache.tools.ant.taskdefs.Manifest(manifestReader);
>- classpath
>- = manifest.getMainSection().getAttributeValue("Class-Path");
>+ classpath = (String) pathMap.get(absPathPlusTimeAndLength);
>+ if (classpath == null) {
>+ ZipFile jarFile = null;
>+ InputStream manifestStream = null;
>+ try {
>+ jarFile = new ZipFile(pathComponent);
>+ manifestStream
>+ = jarFile.getInputStream(new ZipEntry("META-INF/MANIFEST.MF"));
>+
>+ if (manifestStream == null) {
>+ return;
>+ }
>+ Reader manifestReader
>+ = new InputStreamReader(manifestStream, "UTF-8");
>+ org.apache.tools.ant.taskdefs.Manifest manifest
>+ = new org.apache.tools.ant.taskdefs.Manifest(manifestReader);
>+ classpath
>+ = manifest.getMainSection().getAttributeValue("Class-Path");
>
>- } catch (org.apache.tools.ant.taskdefs.ManifestException e) {
>- // ignore
>- } finally {
>- if (manifestStream != null) {
>- manifestStream.close();
>+ } catch (org.apache.tools.ant.taskdefs.ManifestException e) {
>+ // ignore
>+ } finally {
>+ if (manifestStream != null) {
>+ manifestStream.close();
>+ }
>+ if (jarFile != null) {
>+ jarFile.close();
>+ }
> }
>- if (jarFile != null) {
>- jarFile.close();
>+ if (classpath == null) {
>+ classpath = "";
> }
>+ pathMap.put(absPathPlusTimeAndLength, classpath);
> }
>
>- if (classpath != null) {
>+ if (!"".equals(classpath)) {
> URL baseURL = fileUtils.getFileURL(pathComponent);
> StringTokenizer st = new StringTokenizer(classpath);
> while (st.hasMoreTokens()) {
>
>
>
>------------------------------------------------------------------------
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
>For additional commands, e-mail: dev-help@ant.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: New Launcher and JPackage RPMs
Posted by Peter Reilly <pe...@corvil.com>.
Antoine L�vy-Lambert wrote:
> Hi Peter,
>
> can we improve the new classloader so that the work of inspecting the
> jars does not happen each time an <antcall/> is done ?
Yes by keeping a static map of file->manifest class path.
With the following:
<project default="call-all">
<taskdef resource="net/sf/antcontrib/antcontrib.properties" />
<taskdef resource="cpptasks.tasks"/>
<typedef resource="cpptasks.types"/>
<target name="call-all">
<foreach list="1,2,3,4,5,6,7,8,9,10" param="name" target="doit"/>
<echo>Done</echo>
</target>
<target name="doit">
<foreach list="1,2,3,4,5,6,7,8,9,10" param="name" target="doit2"/>
</target>
<target name="doit2"/>
</project>
and the included patch, the time is reduced from 6 seconds to 3 seconds
- not
a great improvement considering that there is 100 ant-calls, but it is a
improvement!
Peter
Re: New Launcher and JPackage RPMs
Posted by Antoine Lévy-Lambert <an...@antbuild.com>.
Hi Peter,
can we improve the new classloader so that the work of inspecting the
jars does not happen each time an <antcall/> is done ?
Antoine
Peter Reilly wrote:
> Stefan Bodewig wrote:
>
>> Hi,
>>
>> today I found myself discussing with the JPackage folks on their list
>> about the problems the RPM setup has with the new Launcher.
>>
>> They put all the installed jars into a single directory and let config
>> files put together the classpath before they invoke Ant IIUC.
>>
>> So far we've told people that the new Launcher would pull in all jars
>> from ANT_HOME/lib (and in addition from ~/.ant/lib, but we didn't say
>> that very loud) - but this is not true. It pulls in all jars from the
>> directory containing ant-launcher.jar, no matter where it is.
>>
>> In stock installs this is the same, but not in the RPM installation
>> where this behavior pulls in the whole Java universe.
>>
>>
> This may have performance implications.
> I was looking at slowdowns from 1.5 to 1.6. One issue is
> <typedef resource="cpptasks.tasks"/>
> This is slower in 1.6 than it is in 1.5. This is due to the new
> classloader (I think). The new classloader added the jars in
> ${ant.home}/lib to the
> classpath and also opens each jar looking for classpath information in
> it.
> If the <typedef/> is at the top level, this happens each time an
> ant-call is
> done - so a common idiom causes problems:
>
> <project name="project">
> <include name="common.xml"/> ( or &common;) <target
> name="compile-mains">
> <foreach list="a,b,c,d,e,f,g,h,i" param="program"
> target="compile"/>
> </target>
> <target name="compile">
> ....
> </target>
> </project>
>
>
> Where common.xml contains:
> <taskdef resource="net/sf/antcontrib/antcontrib.properties" />
> <taskdef resource="cpptasks.tasks"/>
> <typedef resource="cpptasks.types"/>
>
> Peter
>
>> I'm not convinced that we need to adjust anything, but it is an
>> interesting thing to note.
>>
>> Stefan
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
>> For additional commands, e-mail: dev-help@ant.apache.org
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> For additional commands, e-mail: dev-help@ant.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: New Launcher and JPackage RPMs
Posted by Peter Reilly <pe...@corvil.com>.
Stefan Bodewig wrote:
>Hi,
>
>today I found myself discussing with the JPackage folks on their list
>about the problems the RPM setup has with the new Launcher.
>
>They put all the installed jars into a single directory and let config
>files put together the classpath before they invoke Ant IIUC.
>
>So far we've told people that the new Launcher would pull in all jars
>from ANT_HOME/lib (and in addition from ~/.ant/lib, but we didn't say
>that very loud) - but this is not true. It pulls in all jars from the
>directory containing ant-launcher.jar, no matter where it is.
>
>In stock installs this is the same, but not in the RPM installation
>where this behavior pulls in the whole Java universe.
>
>
This may have performance implications.
I was looking at slowdowns from 1.5 to 1.6. One issue is
<typedef resource="cpptasks.tasks"/>
This is slower in 1.6 than it is in 1.5. This is due to the new
classloader (I think). The new classloader added the jars in
${ant.home}/lib to the
classpath and also opens each jar looking for classpath information in it.
If the <typedef/> is at the top level, this happens each time an ant-call is
done - so a common idiom causes problems:
<project name="project">
<include name="common.xml"/> ( or &common;)
<target name="compile-mains">
<foreach list="a,b,c,d,e,f,g,h,i" param="program" target="compile"/>
</target>
<target name="compile">
....
</target>
</project>
Where common.xml contains:
<taskdef resource="net/sf/antcontrib/antcontrib.properties" />
<taskdef resource="cpptasks.tasks"/>
<typedef resource="cpptasks.types"/>
Peter
>I'm not convinced that we need to adjust anything, but it is an
>interesting thing to note.
>
>Stefan
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
>For additional commands, e-mail: dev-help@ant.apache.org
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org