You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ant.apache.org by Kevin Cline <ke...@gmail.com> on 2006/10/18 20:20:51 UTC
AbstractFileSet optimization
My project has a class path containing about 100 jars stored in a Clearcase
vob. We have a this fileset definition:
<path id ="classpath.unit_test">
<pathelement path="bin"/>
<!-- add all files in lib.dir to path -->
<fileset dir="lib">
<include name="**/*.jar"/>
</fileset>
</path>
which is used to run JUnit tests like this:
<target name="utest">
<junit showoutput="true">
<classpath refid="classpath.unit_test"/>
<formatter type="plain" usefile="false" />
<test name="test1"/>
<test name="test2"/>
...
</junit>
</target>
It takes about 30 seconds for the file set to scan the lib directory for
jars. This happens for each test element.
To fix this, I modified the AbstractFileSet class to save the
DirectoryScanner, instead of rescanning each time:
68a69,70
> private DirectoryScanner scanner = null;
>
355,359c357,365
< DirectoryScanner ds = new DirectoryScanner();
< setupDirectoryScanner(ds, p);
< ds.setFollowSymlinks(followSymlinks);
< ds.scan();
< return ds;
---
>
> if (scanner == null) {
> scanner = new DirectoryScanner();
> setupDirectoryScanner(scanner, p);
> scanner.setFollowSymlinks(followSymlinks);
> scanner.scan();
> }
>
> return scanner;
Can this change be incorporated in the next version of Ant?
Re: AbstractFileSet optimization
Posted by Steve Loughran <st...@apache.org>.
Jeffrey E Care wrote:
> No, it's more about institutional inertia.
>
> For WAS, once you consider programmers, architects, testers, documentation
> writers, managers, the build team, project management, L3 support & anyone
> else who needs access to our change management or source control (because
> CMVC integrates both) you're talking about a lot of people; probably well
> over a 1000 for everyone who needs access for one reason or another.
> Moving to *anything* else would require a tremendous amount of training,
> effort & expense, not to mention the impact to schedules for a product
> that *never* gets any downtime.
>
> The support angle is one that is particularly easy to overlook. It's not
> like WAS is an open-source project that has the luxury of saying "sorry,
> we don't support old versions; try our latest nightly build to see if it
> fixes your problem"; our customers with multi-million dollar support
> contracts would probably take exception to that :-) The question then
> becomes "Do we migrate old releases to the new system?" If not, then a
> significant portion of the collective team needs to retain skills for the
> old systems, and we have to continue to pay to maintain those old systems.
> If you do migrate old releases to the new system you need to do so in a
> way that does not interrupt the delivery of critical fixes or even
> regularly scheduled maintenance to customers.
Well, CC is the only SCN system that I know of where an outage means 'no
work, go home early'. Other tools may stop you checking in, and slowly
your local copy goes out of sync, but when CC starts playing up, well,
you have an immediate crisis on your hands.
Its also very high cost in terms of licenses, support people &c. Now,
you dont have to worry so much about licenses, but you will still need
the FTEs to nurture the repository, to keep it fed with live animals or
whatever.
> Platform support is another killer for us; we need clients on all kinds of
> exotic operating system/processor architecture combinations. Can you get a
> ClearCase client for Solaris on Opteron or HPUX on Itanium? What about a
> Subversion client for zOS or i5OS? Most projects don't have to worry about
> such things, but we do :-)
CC is derived from Apollo, which HP bought, and is the tool that HPUX is
built on. If ever they make a change to the FS that is incompatible with
CC, there was some problems until CC was working again.
I do recall that it didnt work on WinXP when that first came out.
> Throw in the fun we have with satisfying auditors for US FDA &
> CommonCriteria requirements and you can start to see why we're reluctant
> to move off of a known system, even if the grass looks greener on the
> other side. As a student & as a regular developer I never had an
> appreciation for these kinds of issues but I do now.
Makes sense.
-steve
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: AbstractFileSet optimization
Posted by Jeffrey E Care <ca...@us.ibm.com>.
No, it's more about institutional inertia.
For WAS, once you consider programmers, architects, testers, documentation
writers, managers, the build team, project management, L3 support & anyone
else who needs access to our change management or source control (because
CMVC integrates both) you're talking about a lot of people; probably well
over a 1000 for everyone who needs access for one reason or another.
Moving to *anything* else would require a tremendous amount of training,
effort & expense, not to mention the impact to schedules for a product
that *never* gets any downtime.
The support angle is one that is particularly easy to overlook. It's not
like WAS is an open-source project that has the luxury of saying "sorry,
we don't support old versions; try our latest nightly build to see if it
fixes your problem"; our customers with multi-million dollar support
contracts would probably take exception to that :-) The question then
becomes "Do we migrate old releases to the new system?" If not, then a
significant portion of the collective team needs to retain skills for the
old systems, and we have to continue to pay to maintain those old systems.
If you do migrate old releases to the new system you need to do so in a
way that does not interrupt the delivery of critical fixes or even
regularly scheduled maintenance to customers.
Platform support is another killer for us; we need clients on all kinds of
exotic operating system/processor architecture combinations. Can you get a
ClearCase client for Solaris on Opteron or HPUX on Itanium? What about a
Subversion client for zOS or i5OS? Most projects don't have to worry about
such things, but we do :-)
Throw in the fun we have with satisfying auditors for US FDA &
CommonCriteria requirements and you can start to see why we're reluctant
to move off of a known system, even if the grass looks greener on the
other side. As a student & as a regular developer I never had an
appreciation for these kinds of issues but I do now.
____________________________________________________________________________________________
Jeffrey E. (Jeff) Care
carej@us.ibm.com
IBM WebSphere Application Server Development
WAS 7.0 Lead Release Engineer
Steve Loughran <st...@apache.org>
10/19/2006 06:43 AM
Please respond to
"Ant Developers List" <de...@ant.apache.org>
To
Ant Developers List <de...@ant.apache.org>
cc
Subject
Re: AbstractFileSet optimization
Jeffrey E Care wrote:
> Actually, WAS doesn't use ClearCase; we're still on IBM CMVC.
what, you're not scared of it, are you :)
-steve
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: AbstractFileSet optimization
Posted by Steve Loughran <st...@apache.org>.
Jeffrey E Care wrote:
> Actually, WAS doesn't use ClearCase; we're still on IBM CMVC.
what, you're not scared of it, are you :)
-steve
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: AbstractFileSet optimization
Posted by Jeffrey E Care <ca...@us.ibm.com>.
Actually, WAS doesn't use ClearCase; we're still on IBM CMVC.
____________________________________________________________________________________________
Jeffrey E. (Jeff) Care
carej@us.ibm.com
IBM WebSphere Application Server Development
WAS 7.0 Lead Release Engineer
"Dominique Devienne" <dd...@gmail.com> wrote on 10/18/2006 03:10:43
PM:
> > > Might I suggest using pathconvert instead? That way you can
construct the
> > > path once, convert it to a property & then use the property in your
tests.
> >
> > Thanks. That solves my problem.
>
> Darn, I didn't think of that! Simple and efficient.
>
> Jeffrey must be used to this type of work around, since he's at IBM,
> home of ClearCase ;-) I used to work for a company that also used
> VOB's as the (moving) "repository" of build artifacts one depends
> upon. I'm more a proper publish artifacts, and do a explicit local
> update to get them kind-of-guy. --DD
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> For additional commands, e-mail: dev-help@ant.apache.org
>
Re: AbstractFileSet optimization
Posted by Dominique Devienne <dd...@gmail.com>.
> > Might I suggest using pathconvert instead? That way you can construct the
> > path once, convert it to a property & then use the property in your tests.
>
> Thanks. That solves my problem.
Darn, I didn't think of that! Simple and efficient.
Jeffrey must be used to this type of work around, since he's at IBM,
home of ClearCase ;-) I used to work for a company that also used
VOB's as the (moving) "repository" of build artifacts one depends
upon. I'm more a proper publish artifacts, and do a explicit local
update to get them kind-of-guy. --DD
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: AbstractFileSet optimization
Posted by Kevin Cline <ke...@gmail.com>.
On 10/18/06, Jeffrey E Care <ca...@us.ibm.com> wrote:
>
>
> FWIW I'm -1 on this change.
>
> What happens when the contents of the directory changes? With your
> proposed patch new files will not be picked up & deleted files will not be
> removed.
>
> Might I suggest using pathconvert instead? That way you can construct the
> path once, convert it to a property & then use the property in your tests.
Thanks. That solves my problem.
Re: AbstractFileSet optimization
Posted by Jeffrey E Care <ca...@us.ibm.com>.
FWIW I'm -1 on this change.
What happens when the contents of the directory changes? With your
proposed patch new files will not be picked up & deleted files will not be
removed.
Might I suggest using pathconvert instead? That way you can construct the
path once, convert it to a property & then use the property in your tests.
____________________________________________________________________________________________
Jeffrey E. (Jeff) Care
carej@us.ibm.com
IBM WebSphere Application Server Development
WAS 7.0 Lead Release Engineer
"Kevin Cline" <ke...@gmail.com>
10/18/2006 02:20 PM
Please respond to
"Ant Developers List" <de...@ant.apache.org>
To
dev@ant.apache.org
cc
Subject
AbstractFileSet optimization
My project has a class path containing about 100 jars stored in a
Clearcase
vob. We have a this fileset definition:
<path id ="classpath.unit_test">
<pathelement path="bin"/>
<!-- add all files in lib.dir to path -->
<fileset dir="lib">
<include name="**/*.jar"/>
</fileset>
</path>
in
which is used to run JUnit tests like this:
<target name="utest">
<junit showoutput="true">
<classpath refid="classpath.unit_test"/>
<formatter type="plain" usefile="false" />
<test name="test1"/>
<test name="test2"/>
...
</junit>
</target>
It takes about 30 seconds for the file set to scan the lib directory for
jars. This happens for each test element.
To fix this, I modified the AbstractFileSet class to save the
DirectoryScanner, instead of rescanning each time:
68a69,70
> private DirectoryScanner scanner = null;
>
355,359c357,365
< DirectoryScanner ds = new DirectoryScanner();
< setupDirectoryScanner(ds, p);
< ds.setFollowSymlinks(followSymlinks);
< ds.scan();
< return ds;
---
>
> if (scanner == null) {
> scanner = new DirectoryScanner();
> setupDirectoryScanner(scanner, p);
> scanner.setFollowSymlinks(followSymlinks);
> scanner.scan();
> }
>
> return scanner;
Can this change be incorporated in the next version of Ant?