incremental building

[John Camelon <jo...@taralnetworks.com>]

i'm trying really hard to use ant for our build environment, but its tough
to think "differently" than what i've been taught over the long hard years
using make.  :)

one thing that i haven't been able to accomplish/understand is how to
effectively do incremental build.  i have numerous projects that all run off
of one build.xml at the root of the classpath as i have seen recommended off
the list from time to time.  problem is that we use CORBA, and there are
numerous IDL's that require compilation to java files prior to running the
compile.

the inherent timestamping using </tstamp> doesn't determine that these IDL's
don't need to be regenerated, and the optional task <depends> only is
relevant for .java --> .class transformations.  so unfortunately, in order
to get incremental compilation working properly, i have had to separate the
two tasks (idl and compile) and not have them depend upon each other.  i
would like to have compile depend upon idl though, and i would like some
incremental mechanism.

any suggestions or references would be helpful.

johnc

Re: incremental building

[Stefan Bodewig <bo...@apache.org>]

John Fisher <fi...@llnl.gov> wrote:

> We are dealing with the same problem, and unfortunately, the
> solution is not that easy. The main problem is that the generated
> .java file names are derived from the *content* of the .idl files,
> not the filenames of the .idl files.

I knew that it wouldn't be that easy in general - this means you'll
have to write your own mapper that knows how to create target file
names from source file names (note that mappers are not restricted to
map one to one, you can have multiple targets for a single input or a
single target for multiple input files).

With something like this <apply> would be general enough.

Stefan

RE: incremental building

[John Fisher <fi...@llnl.gov>]

We are dealing with the same problem, and unfortunately, the solution is not
that easy. The main problem is that the generated .java file names are
derived from the *content* of the .idl files, not the filenames of the .idl
files. In other words, Foo.java does not come from Foo.idl. Rather, Foo.java
would come from XXX.idl, where XXX.idl contains an *interface* called Foo.
This makes everything quite harder.

But, I've found that for Visibroker, the idl2java tool is actually pretty
smart; it only regenerates files that it needs to generate (i.e., that have
changed). Unfortunately, with a lot of IDL files present, the result is
still rather slow, so I've made an "idl" target, a "compile" target, and a
"build" target (which contains both idl and compile as dependencies).

John



>
> John Camelon <jo...@taralnetworks.com> wrote:
>
> > the inherent timestamping using </tstamp> doesn't determine that
> > these IDL's don't need to be regenerated
>
> I think you are expecting far too much from the tstamp task - it just
> sets a couple of properties initializing them from the current time.
>
> What you really want is either <uptodate> or <apply>.  Assuming an
> idlcompiler named idl2java that creates Foo.java from Foo.idl (totally
> made up, I'm sure things are little bit more complicated at your site,
> but to give you an idea) you could do:
>
> <uptodate property="no-idl-regen" targetfile="Foo.java">
>   <srcfiles dir="." includes="Foo.idl" />
> </uptodate>
>
> <target .... unless="no-idl-regen">
>   run idl2java on Foo.idl
> </target>
>
> or to do this in one pass (mapping all .idl files to .java files and
> processing them in a single task):
>
> <apply executable="idl2java">
>   <srcfile />
>
>   <fileset dir="." includes="**/*.idl" />
>   <mapper type="glob" from="*.idl" to="*.java" />
> </apply>
>
> which will recursively scan your basedir for files with the extension
> .idl, compare their last modification time with the one of the
> corresponding .java file (if present) and run the command if (1) the
> .java file is not there or (2) is older than the .idl file.
>
> Stefan
>

CVS commands...

[Mark Jaffe <mj...@eturn.com>]

Hi, all. I am wondering how people use Ant cvs tools, and if anyone has
suggestions on ways of extracting check-in comments for release notes.

RE: incremental building

[John Camelon <jo...@taralnetworks.com>]

thanks for the answer.  :)

i would be interesting in investigating a mechanism to try and integrate
such a heuristic into an interface that a task could implement, such that
you could have real incremental conditions on any task that wanted it.  this
is a problem that comes up often in the non-java world.

johnc

> -----Original Message-----
> From: Stefan Bodewig [mailto:bodewig@apache.org]
> Sent: Thursday, May 10, 2001 2:36 AM
> To: ant-user@jakarta.apache.org
> Subject: Re: incremental building
>
>
> John Camelon <jo...@taralnetworks.com> wrote:
>
> > the inherent timestamping using </tstamp> doesn't determine that
> > these IDL's don't need to be regenerated
>
> I think you are expecting far too much from the tstamp task - it just
> sets a couple of properties initializing them from the current time.
>
> What you really want is either <uptodate> or <apply>.  Assuming an
> idlcompiler named idl2java that creates Foo.java from Foo.idl (totally
> made up, I'm sure things are little bit more complicated at your site,
> but to give you an idea) you could do:
>
> <uptodate property="no-idl-regen" targetfile="Foo.java">
>   <srcfiles dir="." includes="Foo.idl" />
> </uptodate>
>
> <target .... unless="no-idl-regen">
>   run idl2java on Foo.idl
> </target>
>
> or to do this in one pass (mapping all .idl files to .java files and
> processing them in a single task):
>
> <apply executable="idl2java">
>   <srcfile />
>
>   <fileset dir="." includes="**/*.idl" />
>   <mapper type="glob" from="*.idl" to="*.java" />
> </apply>
>
> which will recursively scan your basedir for files with the extension
> .idl, compare their last modification time with the one of the
> corresponding .java file (if present) and run the command if (1) the
> .java file is not there or (2) is older than the .idl file.
>
> Stefan
>

Re: incremental building

[Stefan Bodewig <bo...@apache.org>]

John Camelon <jo...@taralnetworks.com> wrote:

> the inherent timestamping using </tstamp> doesn't determine that
> these IDL's don't need to be regenerated

I think you are expecting far too much from the tstamp task - it just
sets a couple of properties initializing them from the current time.

What you really want is either <uptodate> or <apply>.  Assuming an
idlcompiler named idl2java that creates Foo.java from Foo.idl (totally
made up, I'm sure things are little bit more complicated at your site,
but to give you an idea) you could do:

<uptodate property="no-idl-regen" targetfile="Foo.java">
  <srcfiles dir="." includes="Foo.idl" />
</uptodate>

<target .... unless="no-idl-regen">
  run idl2java on Foo.idl
</target>

or to do this in one pass (mapping all .idl files to .java files and
processing them in a single task):

<apply executable="idl2java">
  <srcfile />

  <fileset dir="." includes="**/*.idl" />
  <mapper type="glob" from="*.idl" to="*.java" />
</apply>

which will recursively scan your basedir for files with the extension
.idl, compare their last modification time with the one of the
corresponding .java file (if present) and run the command if (1) the
.java file is not there or (2) is older than the .idl file.

Stefan

RE: incremental building

[John Camelon <jo...@taralnetworks.com>]

so nobody has run into anything like this before?
or is this a taboo subject on this list?

johnc

> -----Original Message-----
> From: John Camelon [mailto:johnc@taralnetworks.com]
> Sent: Tuesday, May 08, 2001 2:53 PM
> To: 'Ant-User list (E-mail)
> Subject: incremental building
>
>
> i'm trying really hard to use ant for our build environment, but its tough
> to think "differently" than what i've been taught over the long hard years
> using make.  :)
>
> one thing that i haven't been able to accomplish/understand is how to
> effectively do incremental build.  i have numerous projects that
> all run off
> of one build.xml at the root of the classpath as i have seen
> recommended off
> the list from time to time.  problem is that we use CORBA, and there are
> numerous IDL's that require compilation to java files prior to running the
> compile.
>
> the inherent timestamping using </tstamp> doesn't determine that
> these IDL's
> don't need to be regenerated, and the optional task <depends> only is
> relevant for .java --> .class transformations.  so unfortunately, in order
> to get incremental compilation working properly, i have had to
> separate the
> two tasks (idl and compile) and not have them depend upon each other.  i
> would like to have compile depend upon idl though, and i would like some
> incremental mechanism.
>
> any suggestions or references would be helpful.
>
> johnc
>