Initial impressions of ant...

[Paul Kilroy <pk...@channelpoint.com>]

I'm in the process of migrating our project to ant, everything went smoothly
except the following:

1)I initially tried to write a task that used (w/o inheritance) the Java
task. This was too hard IMO. Had to set the project on the Java task, but
the method was package scoped, so I had to play some silly games.
  -I think the the core tasks should be easily reusable and extendable.
  -Tasks should probably take a project on construction, but this is a big
change. If this can't be done, setProject() should be public.

2) Core tasks should do more parameter checking. On the java task, I set
failonerror to true AND fork to false. This should probably produce an error
/ warning.

3) When I wrote my first custom task, my create method returned null, this
caused an NPE in ant code. Core code interacting with user code should be
more defensive.

4) Java code that is forked has no access to input.
  -There was a TODO in the code for this, I submitted a patch for this to
the mailing list a few hours ago.

5) The main loop in StreamPumper.java (IMO) has some issues:
  -The input stream is not buffered, and could overrun.
  -The loop relys on the InputStreams closing for read() to return. This
works for the sub processes output, but not for its input. I'd prefer
something like:

while( keepReading ) {
  size = available();
  if( size > 0 ) {
    read( buf, size );
    write( buf, size );
  } else {
    sleep( SLEEP );
  }
}

Then the PumpStreamHandler should (gracefully?) request that the
StreamPumper stop reading.

6) ClassLoaders are not cached. There was some discussion on this ealier
related to loading new tasks. I'd like to point out that this could also be
helpful for running java code too. Here is my situation:
  -I start ant w/o anything in the classpath. I read a property file to find
out what my classpath needs to be. I then run a series of 60 small programs
to configure and load my database. Each of these 60 programs needs this new
classpath. Currently (if I'm not mistaken) ant creates 60 ClassLoaders and
loads these classes over and over.
  -I think adding an argument to tasks that take classpaths like "cacheid"
or something, might help this.

Other Questions
---------------

-How can I run a task on a build failure? Do I have to use a listener?

-I thought it would be interesting if a target knew how to "clean" itself
up. This could be specified in the target along with how to "build" it. Any
thoughts?

-I can do most of this work I described above. What is the process for this.
Create a bug, then write a patch? Or just submit a patch? Or something else?

Thanks

Paul

Re: Initial impressions of ant...

[Stefan Bodewig <bo...@apache.org>]

Conor has already covered most of this ...

On Mon, 23 Jul 2001, Paul Kilroy <pk...@channelpoint.com> wrote:

> 5) The main loop in StreamPumper.java (IMO) has some issues:

OK, I'll look into this.

> Then the PumpStreamHandler should (gracefully?) request that the
> StreamPumper stop reading.

Yes.

> -How can I run a task on a build failure? Do I have to use a
> listener?

Yes.

> -I thought it would be interesting if a target knew how to "clean"
> itself up.

Clean up in which situation? Always, if a task that is enclosed in it
fails (if so, the clean up will probably depend on the task that
failed), if any other thing fails after this target has been executed?

We might be setting wrong expectations if we did something like this.

Stefan

Re: Initial impressions of ant...

[Jesse Glick <je...@sun.com>]

Conor MacNeill wrote:
> [snip]
> I'm currently mulling over classloader caching. One concern I have is that,
> certainly on Windows, the classloader will retain read locks on the jar
> files in the classpath. These can be the cause of mysterious "access denied"
> errors when people try to delete these jar files. So caching can be a
> problem too. One possibility would be to create a classloader data type
> [snip]

Following this point only: classloader caching is potentially powerful but I
would be skeptical of it working without explicit user control, via a
classloader data type or similar. Only the user can know for sure whether a
JAR included in the path might be deleted (and even recreated) during the
build, thus whether its loader should be cached or recreated. Of course
exposing classloaders to a user is not especially friendly, but better to be
cumbersome and predictable than slick and occasionally broken.

One trick I found helpful working on NetBeans's JAR-based module system:
create a copy of the JAR as a temporary delete-on-exit file and load from
that. Then you have no problems with locks or corrupted archives. (On Windows
the JAR loaded from cannot be deleted or overwritten; on Unix it can but you
will get random errors if any attempt is made to use the classloader later.)
Of course there is some overhead to make the copy and it can leave garbage
files lying around in the /tmp directory.

Also note that under current JDKs, if you ever load anything from a JAR using
ResourceBundle.getBundle, chances are the classloader will never be collected
and any locks held on the JAR will stay there. Anyway you would have to do a
System.gc to release unused JAR locks, assuming no Class's remain from the
classloader you made. And even this does not work at all on 1.2.2 on Windows
at least, you need to use 1.3 to have a chance at releasing the lock.

-Jesse

-- 
Jesse Glick        <ma...@sun.com>
NetBeans, Open APIs  <http://www.netbeans.org/>
tel (+4202) 3300-9161 Sun Micro x49161 Praha CR

Re: Initial impressions of ant...

[Stefan Bodewig <bo...@apache.org>]

On Mon, 23 Jul 2001, <cm...@yahoo.com> wrote:

> Probably that's something for ant2, but beeing able to use the tasks
> as standalone beans in normal java code would be extremely
> usefull.

I'd prefer to not use the tasks directly - rather have the tasks as a
thin Ant specific layer on top of service code.

For example, we already have the Execute class as a service class you
can use to execute an arbitrary system command and people should use
this sevice class instead of the implementation of the <exec> task.

There are many other tasks that should (and I hope will) be refactored
into a service class more or less independent of Ant and a Ant task
layer on top of it, this includes <javac> (where the service layer
could come from Sam's commons javac package idea) and <java>, but even
rather simple things like <touch> or <copy> as well.

This seems to boil down to: don't use the tasks directly.  If it is
easier to use the task than the underlying service framework (or there
is no separate service), refactor the task - which is something for
Ant2, of course.

Stefan

RE: Initial impressions of ant...

[cm...@yahoo.com]

On Tue, 24 Jul 2001, Conor MacNeill wrote:

> OK. fair view. In Ant 1.x, it is, however, probably not practical. The tasks
> are not cleanly separated from the core in the current implementation. If
> you want to use a particular Task, you are going to need to bring along
> Project, etc, etc. It is highly coupled, unfortunately.
>
> The solution we have considered for Ant2, is to define the interface between
> a task and the core - through a TaskContext. It is similar to Servlet
> concepts and I think it is useful. If you want to use Tasks in another
> context, that will be fine but you are going to need to provide that
> interface to provide the Tasks with the services they require. That will
> decouple the Task from its environment.

> > Probably that's something for ant2, but beeing able to use the tasks as
> > standalone beans in normal java code would be extremely usefull. Even in
> > the current ant, it can be done just by making few methods public and
> > decoupling a bit.
>
> The same concept could be provided in the current Ant, true enough. It would
> be a "backward compatiblity" challenge.

Most work will be to convert the tasks to the new interface, and to
"convert" people who write tasks.

IMHO the sooner the TaskContext interface is added, the easier will be to
do the transition. I know this is a big step and ant2 is the "normal"
place to do it, but trying few small things and adding few interfaces now
could help a lot.

> > Example: javac task is something very usefull outside ant ( jasper, xalan,
> > etc), and having it available ( with Project acting as an execution
> > context - i.e. handle logging, etc ) will give people a more "consistent"
> > experience with different apache projects ( same way to specify "use
> > jikes" or "generate emacs errors ).
>
> That is reasonable. Should the Ant task be the way to package up such
> services? Perhaps. then again, you could packge up the service and build an
> ant task around it.

Yes, that's what always do - I create the beans ( with no dependency on
ant ), then use the task adapter.

I don't use any ant service in my code ( well, sometimes I feel a need for
logging and for a hashtable like refIds, but it's easy to do that
without using ant, but separate tools ).

In other words - it is possible to write ant tasks without any dependency
on ant. Probably not all tasks - but many.

The Task interface is good  to avoid a reflection ( when calling
execute() ), but that's a small gain giving the number of reflection calls
to set properties, etc.


Costin

RE: Initial impressions of ant...

[Conor MacNeill <co...@cortexebusiness.com.au>]

> From: cmanolache@yahoo.com [mailto:cmanolache@yahoo.com]
> Sent: Tuesday, 24 July 2001 11:53 AM
> To: ant-dev@jakarta.apache.org
> Subject: RE: Initial impressions of ant...
>
>
> On Tue, 24 Jul 2001, Conor MacNeill wrote:
>
> > > From: Paul Kilroy [mailto:pkilroy@channelpoint.com]
> > >
> > > I'm in the process of migrating our project to ant, everything
> > > went smoothly
> > > except the following:
> > >
> > > 1)I initially tried to write a task that used (w/o
> inheritance) the Java
> > > task. This was too hard IMO. Had to set the project on the
> Java task, but
> > > the method was package scoped, so I had to play some silly games.
> >
> > You should probably use project.createTask("java"). This will properly
> > initialise the Task. I guess it is a bit like servlets in a servlet
> > container. You can't just create them and hope they will be
> connected into
> > the container's infrastructure. It is the same in Ant.
>
> That doesn't mean ant should do the same :-)

OK. fair view. In Ant 1.x, it is, however, probably not practical. The tasks
are not cleanly separated from the core in the current implementation. If
you want to use a particular Task, you are going to need to bring along
Project, etc, etc. It is highly coupled, unfortunately.

The solution we have considered for Ant2, is to define the interface between
a task and the core - through a TaskContext. It is similar to Servlet
concepts and I think it is useful. If you want to use Tasks in another
context, that will be fine but you are going to need to provide that
interface to provide the Tasks with the services they require. That will
decouple the Task from its environment.

>
> Probably that's something for ant2, but beeing able to use the tasks as
> standalone beans in normal java code would be extremely usefull. Even in
> the current ant, it can be done just by making few methods public and
> decoupling a bit.

The same concept could be provided in the current Ant, true enough. It would
be a "backward compatiblity" challenge.

>
> Example: javac task is something very usefull outside ant ( jasper, xalan,
> etc), and having it available ( with Project acting as an execution
> context - i.e. handle logging, etc ) will give people a more "consistent"
> experience with different apache projects ( same way to specify "use
> jikes" or "generate emacs errors ).

That is reasonable. Should the Ant task be the way to package up such
services? Perhaps. then again, you could packge up the service and build an
ant task around it.

>
> ( I know this "framework/simple build tool and nothing else/collection of
> tasks/etc" is a sensitive issue, don't want to reopen it )
>
> > that guarantee should be made in the services not the tasks
> layered on these
> > services. We need to control the scope of the API Ant makes available to
> > task writers to make upgrading Ant manageable.
>
> +1
> And also makes integrating/reusing ant tasks in other places much easier.
> I think the big collection of usefull tasks ( as components ) is as
> important as the "build tool" functionality.

I think that is the approach Peter has championed, and, using the
TaskContext as the interface contract, I too think it is a good way to go.

RE: Initial impressions of ant...

[cm...@yahoo.com]

On Tue, 24 Jul 2001, Conor MacNeill wrote:

> > From: Paul Kilroy [mailto:pkilroy@channelpoint.com]
> >
> > I'm in the process of migrating our project to ant, everything
> > went smoothly
> > except the following:
> >
> > 1)I initially tried to write a task that used (w/o inheritance) the Java
> > task. This was too hard IMO. Had to set the project on the Java task, but
> > the method was package scoped, so I had to play some silly games.
>
> You should probably use project.createTask("java"). This will properly
> initialise the Task. I guess it is a bit like servlets in a servlet
> container. You can't just create them and hope they will be connected into
> the container's infrastructure. It is the same in Ant.

That doesn't mean ant should do the same :-)

Probably that's something for ant2, but beeing able to use the tasks as
standalone beans in normal java code would be extremely usefull. Even in
the current ant, it can be done just by making few methods public and
decoupling a bit.

Example: javac task is something very usefull outside ant ( jasper, xalan,
etc), and having it available ( with Project acting as an execution
context - i.e. handle logging, etc ) will give people a more "consistent"
experience with different apache projects ( same way to specify "use
jikes" or "generate emacs errors ).

( I know this "framework/simple build tool and nothing else/collection of
tasks/etc" is a sensitive issue, don't want to reopen it )

> that guarantee should be made in the services not the tasks layered on these
> services. We need to control the scope of the API Ant makes available to
> task writers to make upgrading Ant manageable.

+1
And also makes integrating/reusing ant tasks in other places much easier.
I think the big collection of usefull tasks ( as components ) is as
important as the "build tool" functionality.


Costin

RE: Initial impressions of ant...

[Conor MacNeill <co...@cortexebusiness.com.au>]

> From: Paul Kilroy [mailto:pkilroy@channelpoint.com]
>
> I'm in the process of migrating our project to ant, everything
> went smoothly
> except the following:
>
> 1)I initially tried to write a task that used (w/o inheritance) the Java
> task. This was too hard IMO. Had to set the project on the Java task, but
> the method was package scoped, so I had to play some silly games.

You should probably use project.createTask("java"). This will properly
initialise the Task. I guess it is a bit like servlets in a servlet
container. You can't just create them and hope they will be connected into
the container's infrastructure. It is the same in Ant.

>   -I think the the core tasks should be easily reusable and extendable.

There are services for executing java programs within Ant (ExecuteJava) and
these should be used in preference to configuring a task. I believe you
should be using these services. Reusable and extendable tasks are nice but
it also creates an ongoing commitment to backward compatability. I think
that guarantee should be made in the services not the tasks layered on these
services. We need to control the scope of the API Ant makes available to
task writers to make upgrading Ant manageable.

Of course, many times it is nicer to use the Task interface, but that is a
failing of the underlying services

>   -Tasks should probably take a project on construction, but this is a big
> change. If this can't be done, setProject() should be public.

I would not want to see a setProject that is public. It may create
expectations that a task could be moved to another project at will. This is
not so.

>
> 2) Core tasks should do more parameter checking. On the java task, I set
> failonerror to true AND fork to false. This should probably
> produce an error
> / warning.

Why? Just because the current code always returns an error code of 0, does
not mean this is a valid combination of attributes. In fact, I have been
working on a patch to capture System.exit calls and return the status value
even for non forked code. This would make this combination valid.

>
> 3) When I wrote my first custom task, my create method returned null, this
> caused an NPE in ant code. Core code interacting with user code should be
> more defensive.

Agreed

>
> 4) Java code that is forked has no access to input.
>   -There was a TODO in the code for this, I submitted a patch for this to
> the mailing list a few hours ago.

OK, I'll check it out.

> 6) ClassLoaders are not cached. There was some discussion on this ealier
> related to loading new tasks. I'd like to point out that this
> could also be
> helpful for running java code too. Here is my situation:
>   -I start ant w/o anything in the classpath. I read a property
> file to find
> out what my classpath needs to be. I then run a series of 60
> small programs
> to configure and load my database. Each of these 60 programs
> needs this new
> classpath. Currently (if I'm not mistaken) ant creates 60 ClassLoaders and
> loads these classes over and over.

I'm currently mulling over classloader caching. One concern I have is that,
certainly on Windows, the classloader will retain read locks on the jar
files in the classpath. These can be the cause of mysterious "access denied"
errors when people try to delete these jar files. So caching can be a
problem too. One possibility would be to create a classloader data type

<classloader id="common">
    <classpath refid="blah"/>
</classloader>

<java>
   <classloader refid="common">
</java>

let me think about that a bit more.

>   -I think adding an argument to tasks that take classpaths like "cacheid"
> or something, might help this.
>
> Other Questions
> ---------------
>
> -How can I run a task on a build failure? Do I have to use a listener?

You currently can't run a task on build failure. We have talked about this
previously. Check the archives. Probably we can discuss this again.

>
> -I thought it would be interesting if a target knew how to "clean" itself
> up. This could be specified in the target along with how to
> "build" it. Any
> thoughts?

I don't like it much.

>
> -I can do most of this work I described above. What is the
> process for this.
> Create a bug, then write a patch? Or just submit a patch? Or
> something else?
>

It depends on you to some extent.
I like it when people create a bug report as it can be tracked - It doesn't
get forgotten. Nevertheless, patches to the dev-list are also effective.

Checkout these links for a guide to how things work
http://jakarta.apache.org/site/getinvolved.html

This link covers patch formats, etc
http://jakarta.apache.org/site/source.html

Cheers
Conor