Looking for someone to implement this idea

[King Dale <Ki...@tce.com>]

I have come up with an idea for some tasks that would actually turn Ant into
a usefull build system for Java code. As it stands now, I don't think Ant is
viable because it has very little support for dependency checking.

The javac will only check dependencies between a source file and a class
file with the same name in the same directory structure. The dependency
checking is a simple is the source file newer than the class file. It will
not rebuild other classes that depend on the source file that changed.

The depend task does a little better. It will do some dependency checking
between classes, but has some holes. It expects the source files to be in
the same directory structure as the packages and that they a class is
generated by a source file of the same name. It doesn't handle inner classes
or static dependencies that are inlined. And the dependency checking is also
using > for timestamps.

The bottom line is that it is possible to have Ant say that it rebuilt
everything yet the software is in an inconsistent state and may not run. In
my book if you ever have to do a make clean to guarantee a correct build
there is something wrong with your build tool. Unfortunately make also
suffers from some of the same problems.

But I believe there is a solution. I know what the solution is, but
unfortunately I do not have the time to write it myself. I was hoping that I
could find some people willing to expend the effort to write it for me and
the good of Ant users everywhere.

What I envision is probably a set of tasks that are somewhat related. Here
is a list of the primary features I expect it to have:

- Ability to handle source files that are not in directories according to
package. Ideally you would like the source files to be in the package
hierarchy, but there is no reason to require it.
- Ability to handle static dependencies, which do not show up in the class
file.
- Ability to handle classes defined in source files that have a different
name.
- Ability to rebuild when the source file length changes or its timestamp
changes (including becoming older), not just when the date is newer than the
class files. Sometimes the timestamp becomes older, like when checking out a
previous release to fix a bug.
- Ability to handle dependencies for inner and nested classes.
- Ability to determine for a given set of source files which files are
compiled from those sources, including filtering based on whether a class
extends a certain class or implements a certain interface. Useful for
determining which files to include in a jar.
- Ability to determine for a given set of files all of the class files that
are used by the class files generated.
- Rebuilding when a jar file in the classpath changes.
- Dependency information should be cached for the next build.

Here is the way it would work to the user. There would be one task that
accepts a fileset of java source files and also a path to a file that will
store the cached dependency information. The task will compile any of the
source files that need to be recompiled and update the dependency
information.

There would be another task or maybe two tasks that would be used to query
some information from that dependency cache. It would take file sets of
source files and could generate a fileset of class files that are built from
those source files (including inner class files). You could filter based
superclass, etc. Since Ant doesn't really have the concept of a task
generating a fileset what you would do is provide an id to be given to the
fileset and the task would create a fileset and store it in the project with
that ID. You could then retrieve it in another task using the ID as a refid.

What may be another task or part of the same task is to also generate a
fileset of those class files that are needed by the generated class files.

So how would this work internally? First off the compilation would have to
be done, because it supports the generation of a dependency report using the
+DR option. That dependency report is the key to the whole thing. That
dependency report tells you:
- Which classes are generated from which source file.
- which classes those generated files depend upon.
- and which classes were retrieved from what jar files.

The dependency cache file will not have the same format, but some of the
information will come from the dependency report. It will also contain
information from the file system and from parsing the generated class files.
The dependency cache file will contain the following information:

- For each source file, jar file, and generated class file it will contain
its canonical path name, its last modified time, and its length. A file will
be considered changed if its last modified time or its length differs from
what is in the dependency cache or if the file does not appear in the
dependency cache (e.g this is the first time we are building).
- For each source file the list of classes it generates and the reverse
mapping from class to source file (if it is a class that is generated from
source).
- For classes retrieved from jar files the mappings between the class and
the jar file.
- For generated classes the list of files containing that class. That is
files plural to handle inner classes. The file names for inner classes is
determined by parsing generated class files and using the InnerClasses
attribute in the class file. All generated classes would have to be scanned
from inner classes.
- Generated class files could also be scanned for superclass and
superinterface information. This would also entail scanning any classes
contained in jar files which are superclasses and superinterfaces. You would
generate for each class the set of classes and interfaces it extends. This
could be an optional step to save time.
- Compiler options used by jikes that could affect classfile generation. If
any of the options change all files should be rebuilt.

As I see it these would be some of the steps in the task:

- Get a list of all files in the fileset.
- Get the length and last modified time of all files in the input.
- If any of the files do not exist that is a build error.
- Read the dependency cache.
- For all files referenced get their length and last modified time.
- If a souce file referenced by the dependency cache no longer exists be
sure to prune its dependency information. Any files that depended on it will
have to be compiled. This is to handle things like when a class is renamed
or deleted.
- Determine given the dependency information, timestamps, lengths and file
sets which files need to be compiled. While this may take some thought in
getting all of this right all the information is there to do this right.
There are certain situations that may cause errors to be generated such as a
file needs to be recompiled that still exists but it is not in the input
file set.
- Compile all files that need to be compiled. The compilation will be done
with jikes and with the +DR option.
- Parse the dependency report and update the dependency information.
- Update the timestamp and length information as necessary.
- Parse the class files for inner class information and optionally
superclass and superinterface information.

All of this is certainly doable, but will require some work. I can help
guide the effort, but do not have the time to do the work myself. Is anyone
interested in working on a project like this?