Incremental build support

[Gu Dev <de...@yahoo.com>]

Hi.
I have allready sent this message but I 
believe it wasn't delivered, so i try again.
 
I am currently working on a very large
project and among other, I am in charge
of the build process in which we use ant.
I really like ant and its ideas and I've
decided to get involved in the project.
 
During the development of our project, the
issue of incremental build had been brought
up several times while using ant. In my
opinion - it is one of ant's biggest flaws.
I'll explain.
 
We maintain all the build process as ant
files in order to run nightly builds. The
problem is that the build process is not
applicable for a developer who had changed
two-three files and want to run the project
and test his/her changes. The full build
process is highly time consuming - which
is ok for night builds, yet unbearable for
developers. The catch is that we don't want
to re-write the building process in our IDE
in order not to increase maintenance time
and error probability of our building
process. As for now, we maintain the
classpaths and more of the build process
(packing..) in two places - in the build
file and in the IDE.
 
My suggestion is to add ant a support
for incremental build in the task level.
Since the current state is that tasks
do not support incremental build, this
will remain the default. A -incremental
option will be supported as an argument
to ant, and then every task will be
able to test the incremental flag of
the project, and behave accordingly.
For example - the copy task will behave
under incremental build as follows - it
will copy the files only if they do not
exist or if they are "younger" at the
target dir. Another example - the jar
task will behave under incremental build
as follows - it will touch the file
only if one of the source files for
the jar is "older" than the jar itself
(pitfall - if the build process will
use the same jar file name to create
two different archives - by the second
jaring the jar will be assumed as newer
than its source files and therefore will
not be packed). There are problems with
adding support to all the tasks (for
example delete) but it seems that it all
can be solved elegantly (as for the
delete problem, using sync task instead
of delete and then copy will allow adding
the incremental support to the sync task).
 
The beauty is that after adding support
for the -incremental option, it does not
affect the tasks immediately, and instead
you can go over it one by one, slow and
easy, and add the incremental support. In
other words - the change can be done with
out adding the support to any task and can
be a part of the new version even without
task support - which makes this change much
more subtle.
 
The advantages are that ant will become
a tool also for the developers, and the project will
have a single point of build definition
rather than two or more. Ant was said to
have all the advantages of makefiles, but
the lack of incremental support is definitely
an advantage of makefiles which I miss the most.
 
What do you say?

Gu Dev.


---------------------------------
Do you Yahoo!?
Free online calendar with sync to Outlook(TM).

Re: Incremental build support

[Gu Dev <de...@yahoo.com>]

You are right about our destructive deleting.
I will join it into a "clean" target. Thanks
for the advice.

I am still wondering if a build process needs
to be "aware" of the fact that it is running in
incremental mode and to what extent is it good 
to leave the build process design to the writer
of the build file. I am a good example for someone
who read all of the documentation and checked some
build file examples, and still produced a pour
design. 

It definiteley can be resolved by posting 
questions and getting the answers as i did, 
but it can also be resolved by adding a 
feature that will spare the build writer from
making a these errors.

Gu Dev.

--- Kris Bravo <kr...@corridor-software.us>
wrote:
> 
> It might be worth it to post at least part of your
> build script to the 
> list to help explain your problem.
> 
> I suggest that because I might be misunderstanding
> what you are saying. 
> Are you suggesting that Ant doesn't
> compile/move/copy/jar, etc only 
> what's necessary (i.e., stale files)? Because it
> does, and your build 
> script might be deleting the effort of the previous
> builds and causing 
> everything to be done from scratch.
> 
> If that is in fact the case, then the issue is that
> your script does 
> destructive builds, not that Ant doesn't only update
> whats necessary. If 
> you can isolate your deleting to "clean" targets,
> then your automated 
> build can call clean, then build and your developers
> can just build.
> 
> Let us have a look at the significant parts of the
> script. If you aren't 
> permitted to post sanitized parts of your build by
> your company, just 
> examine the script for deletes and try to
> consolidate them into a target 
> which only the nightly build calls.
> 
> -Bravo
> 
> Gu Dev wrote:
> 
> >Hi.
> >I have allready sent this message but I 
> >believe it wasn't delivered, so i try again.
> > 
> >I am currently working on a very large
> >project and among other, I am in charge
> >of the build process in which we use ant.
> >I really like ant and its ideas and I've
> >decided to get involved in the project.
> > 
> >During the development of our project, the
> >issue of incremental build had been brought
> >up several times while using ant. In my
> >opinion - it is one of ant's biggest flaws.
> >I'll explain.
> > 
> >We maintain all the build process as ant
> >files in order to run nightly builds. The
> >problem is that the build process is not
> >applicable for a developer who had changed
> >two-three files and want to run the project
> >and test his/her changes. The full build
> >process is highly time consuming - which
> >is ok for night builds, yet unbearable for
> >developers. The catch is that we don't want
> >to re-write the building process in our IDE
> >in order not to increase maintenance time
> >and error probability of our building
> >process. As for now, we maintain the
> >classpaths and more of the build process
> >(packing..) in two places - in the build
> >file and in the IDE.
> > 
> >My suggestion is to add ant a support
> >for incremental build in the task level.
> >Since the current state is that tasks
> >do not support incremental build, this
> >will remain the default. A -incremental
> >option will be supported as an argument
> >to ant, and then every task will be
> >able to test the incremental flag of
> >the project, and behave accordingly.
> >For example - the copy task will behave
> >under incremental build as follows - it
> >will copy the files only if they do not
> >exist or if they are "younger" at the
> >target dir. Another example - the jar
> >task will behave under incremental build
> >as follows - it will touch the file
> >only if one of the source files for
> >the jar is "older" than the jar itself
> >(pitfall - if the build process will
> >use the same jar file name to create
> >two different archives - by the second
> >jaring the jar will be assumed as newer
> >than its source files and therefore will
> >not be packed). There are problems with
> >adding support to all the tasks (for
> >example delete) but it seems that it all
> >can be solved elegantly (as for the
> >delete problem, using sync task instead
> >of delete and then copy will allow adding
> >the incremental support to the sync task).
> > 
> >The beauty is that after adding support
> >for the -incremental option, it does not
> >affect the tasks immediately, and instead
> >you can go over it one by one, slow and
> >easy, and add the incremental support. In
> >other words - the change can be done with
> >out adding the support to any task and can
> >be a part of the new version even without
> >task support - which makes this change much
> >more subtle.
> > 
> >The advantages are that ant will become
> >a tool also for the developers, and the project
> will
> >have a single point of build definition
> >rather than two or more. Ant was said to
> >have all the advantages of makefiles, but
> >the lack of incremental support is definitely
> >an advantage of makefiles which I miss the most.
> > 
> >What do you say?
> >
> >Gu Dev.
> >
> >
> >---------------------------------
> >Do you Yahoo!?
> >Free online calendar with sync to Outlook(TM).
> >  
> >
> 
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> dev-unsubscribe@ant.apache.org
> For additional commands, e-mail:
> dev-help@ant.apache.org
> 


__________________________________
Do you Yahoo!?
Yahoo! Calendar - Free online calendar with sync to Outlook(TM).
http://calendar.yahoo.com

Re: Incremental build support

[Kris Bravo <kr...@corridor-software.us>]

It might be worth it to post at least part of your build script to the 
list to help explain your problem.

I suggest that because I might be misunderstanding what you are saying. 
Are you suggesting that Ant doesn't compile/move/copy/jar, etc only 
what's necessary (i.e., stale files)? Because it does, and your build 
script might be deleting the effort of the previous builds and causing 
everything to be done from scratch.

If that is in fact the case, then the issue is that your script does 
destructive builds, not that Ant doesn't only update whats necessary. If 
you can isolate your deleting to "clean" targets, then your automated 
build can call clean, then build and your developers can just build.

Let us have a look at the significant parts of the script. If you aren't 
permitted to post sanitized parts of your build by your company, just 
examine the script for deletes and try to consolidate them into a target 
which only the nightly build calls.

-Bravo

Gu Dev wrote:

>Hi.
>I have allready sent this message but I 
>believe it wasn't delivered, so i try again.
> 
>I am currently working on a very large
>project and among other, I am in charge
>of the build process in which we use ant.
>I really like ant and its ideas and I've
>decided to get involved in the project.
> 
>During the development of our project, the
>issue of incremental build had been brought
>up several times while using ant. In my
>opinion - it is one of ant's biggest flaws.
>I'll explain.
> 
>We maintain all the build process as ant
>files in order to run nightly builds. The
>problem is that the build process is not
>applicable for a developer who had changed
>two-three files and want to run the project
>and test his/her changes. The full build
>process is highly time consuming - which
>is ok for night builds, yet unbearable for
>developers. The catch is that we don't want
>to re-write the building process in our IDE
>in order not to increase maintenance time
>and error probability of our building
>process. As for now, we maintain the
>classpaths and more of the build process
>(packing..) in two places - in the build
>file and in the IDE.
> 
>My suggestion is to add ant a support
>for incremental build in the task level.
>Since the current state is that tasks
>do not support incremental build, this
>will remain the default. A -incremental
>option will be supported as an argument
>to ant, and then every task will be
>able to test the incremental flag of
>the project, and behave accordingly.
>For example - the copy task will behave
>under incremental build as follows - it
>will copy the files only if they do not
>exist or if they are "younger" at the
>target dir. Another example - the jar
>task will behave under incremental build
>as follows - it will touch the file
>only if one of the source files for
>the jar is "older" than the jar itself
>(pitfall - if the build process will
>use the same jar file name to create
>two different archives - by the second
>jaring the jar will be assumed as newer
>than its source files and therefore will
>not be packed). There are problems with
>adding support to all the tasks (for
>example delete) but it seems that it all
>can be solved elegantly (as for the
>delete problem, using sync task instead
>of delete and then copy will allow adding
>the incremental support to the sync task).
> 
>The beauty is that after adding support
>for the -incremental option, it does not
>affect the tasks immediately, and instead
>you can go over it one by one, slow and
>easy, and add the incremental support. In
>other words - the change can be done with
>out adding the support to any task and can
>be a part of the new version even without
>task support - which makes this change much
>more subtle.
> 
>The advantages are that ant will become
>a tool also for the developers, and the project will
>have a single point of build definition
>rather than two or more. Ant was said to
>have all the advantages of makefiles, but
>the lack of incremental support is definitely
>an advantage of makefiles which I miss the most.
> 
>What do you say?
>
>Gu Dev.
>
>
>---------------------------------
>Do you Yahoo!?
>Free online calendar with sync to Outlook(TM).
>  
>