[Patch] Target attribute property expansion

[Peter Vogel <pv...@arsin.com>]

This is a re-submit based on Stefan's comments.  It expands only the depends
and description attributes,
and includes some documentation updates as well.

-Peter

 <<targattrs.diff>> 

Re: [Patch] Target attribute property expansion

[Stefan Bodewig <bo...@apache.org>]

Peter Vogel <pv...@arsin.com> wrote:

> This is a re-submit based on Stefan's comments.

Thanks, you've seen the reasons why I feel, I cannot commit it.

Stefan

Re: [Patch] Target attribute property expansion

[Stefan Bodewig <bo...@apache.org>]

Thanks for your explanation, yes, what you propose should work.  One
problem we'd need to consider though: If we'd allow arbitrary task to
live outside of targets and execute them before we examine the
dependencies - these tasks would even be executed in a completely
invalid build process.

On Thu, 5 Jul 2001, Peter Donald <do...@apache.org> wrote:

> On Thu,  5 Jul 2001 18:24, Stefan Bodewig wrote:
>> On Thu, 5 Jul 2001, Peter Donald <do...@apache.org> wrote:
> 
>> > This is for two reasons mainly;
>>
>> You mention three 8-)
> 
> so I failed preschool maths ;)

No, my interpretation: if it is for two reasons _mainly_ and you state
three, one of them has to be less important 8-) I guess I know which
one is not part of "mainly".

>> > 1. Support crazy self-modifying projects like Jose wanted
>>
>> He did?
> 
> his projectref ideas.

Never looked like self-modifying projects to me.

>> > 2. Not force the loading of all project files at startup
>>
>> If we allow targets in the main file to depend on targets in other
>> files, we'll have to load and parse this other file anyway, don't
>> we?
> 
> nope - not if they are never executed.

Yes, OK.  If we don't care for targets we never reach, there is no
reason to load that file unless we need it.  Fine with me.

Stefan

Re: [Patch] Target attribute property expansion

[Peter Donald <do...@apache.org>]

On Thu,  5 Jul 2001 18:24, Stefan Bodewig wrote:
> On Thu, 5 Jul 2001, Peter Donald <do...@apache.org> wrote:
> > I actually think that we should do lazy evaluation of the DAG. ie
> > only look up next element in DAG as we "get" to it.
>
> Could you please explain how this should work?  I'm not sure I know
> what you mean by "look up" and "get to" in this context.

Well just say user passes in target foo on command line. It depends on target 
bar and bar depends on baz. So first we look up foo. It is present so there 
is no error. We find it's dependencies (just bar) and look that up. We find 
target bar and it is present. We then lookup it's dependencies (ie baz) and 
find they are not present so we trigger an error there and not at 
"precompile" stage.

We also keep a stack of path to current current target so that we can detec 
circular dependencies etc.

> > This is for two reasons mainly;
>
> You mention three 8-)

so I failed preschool maths ;)

>
> > 1. Support crazy self-modifying projects like Jose wanted
>
> He did?

his projectref ideas.

> > 2. Not force the loading of all project files at startup
>
> If we allow targets in the main file to depend on targets in other
> files, we'll have to load and parse this other file anyway, don't we?

nope - not if they are never executed. So if target foo depend on project1s 
target bar but target foo is never "executed" then we shouldn't have to load 
and parse project1.

This is essentially designed for large projects. I was talking to some peeps 
how they arrange a sprawling build system for a massive project and that was 
one of the optimizations that was essential for them to achieve reasonable 
build times.

> > 3. Cutdown on redundent scanning at start that is not needed and
> > costly in large projects
>
> Scanning for what?  If you are talking about not searching for cycles
> in the dependency graph that we are never going to touch, that is fine
> with me.

That aswell as scanning for existence. For instance if target foo is never 
called then we shouldn't be checking it's dependencies are OK.

Cheers,

Pete

*-----------------------------------------------------*
| "Faced with the choice between changing one's mind, |
| and proving that there is no need to do so - almost |
| everyone gets busy on the proof."                   |
|              - John Kenneth Galbraith               |
*-----------------------------------------------------*

Re: [Patch] Target attribute property expansion

[Glenn McAllister <gl...@somanetworks.com>]

Peter Donald wrote:

> On Thu,  5 Jul 2001 18:50, Jose Alberto Fernandez wrote:
> > Do we want to find as many problems on the build file as possible, or do we
> > want to wait until it breaks once the file is on the production
> > environment? I usually prefer to catch as many errors (problems) as
> > possible as eager as possible. MAybe we should have an option for that.
>
> I have been thinking of adding an antdoc and antlint style system. One would
> doc the buildfile structure while the other would try to check it for
> correctness to the best of it's ability.
>

This strikes me as a reasonable compromise.  I could live with that, and I do
like the idea of "lazy dependencies."

Glenn McAllister
SOMA Networks, Inc.

Re: [Patch] Target attribute property expansion

[Peter Donald <do...@apache.org>]

On Thu,  5 Jul 2001 18:50, Jose Alberto Fernandez wrote:
> Do we want to find as many problems on the build file as possible, or do we
> want to wait until it breaks once the file is on the production
> environment? I usually prefer to catch as many errors (problems) as
> possible as eager as possible. MAybe we should have an option for that.

I have been thinking of adding an antdoc and antlint style system. One would 
doc the buildfile structure while the other would try to check it for 
correctness to the best of it's ability. 

Cheers,

Pete

*-----------------------------------------------------*
| "Faced with the choice between changing one's mind, |
| and proving that there is no need to do so - almost |
| everyone gets busy on the proof."                   |
|              - John Kenneth Galbraith               |
*-----------------------------------------------------*

RE: [Patch] Target attribute property expansion

[Jose Alberto Fernandez <j_...@yahoo.com>]

> From: Stefan Bodewig [mailto:bodewig@apache.org]
>
> On Thu, 5 Jul 2001, Peter Donald <do...@apache.org> wrote:
>
> > I actually think that we should do lazy evaluation of the DAG. ie
> > only look up next element in DAG as we "get" to it.
>
> Could you please explain how this should work?  I'm not sure I know
> what you mean by "look up" and "get to" in this context.
>
> > This is for two reasons mainly;
>
> You mention three 8-)
>
> > 1. Support crazy self-modifying projects like Jose wanted
>
> He did?
>

I do not think I did. Unless he is referring to the following pattern that I
proposed:

	<target ....>
	  <generate-suproject-from-template file="xyz.xml" />
	  <ant location="xyz.xml" .... />
	</target>

which I do not think is self-modifying in the sense of the discussion here,
nor it is crazy, IMHO.

> > 2. Not force the loading of all project files at startup
>
> If we allow targets in the main file to depend on targets in other
> files, we'll have to load and parse this other file anyway, don't we?
>
> > 3. Cutdown on redundent scanning at start that is not needed and
> > costly in large projects
>
> Scanning for what?  If you are talking about not searching for cycles
> in the dependency graph that we are never going to touch, that is fine
> with me.
>

Do we want to find as many problems on the build file as possible, or do we
want to wait until it breaks once the file is on the production environment?
I usually prefer to catch as many errors (problems) as possible as eager as
possible. MAybe we should have an option for that.

Jose Alberto

> Stefan
>

Re: [Patch] Target attribute property expansion

[Stefan Bodewig <bo...@apache.org>]

On Thu, 5 Jul 2001, Peter Donald <do...@apache.org> wrote:

> I actually think that we should do lazy evaluation of the DAG. ie
> only look up next element in DAG as we "get" to it.

Could you please explain how this should work?  I'm not sure I know
what you mean by "look up" and "get to" in this context.

> This is for two reasons mainly;

You mention three 8-)

> 1. Support crazy self-modifying projects like Jose wanted

He did?

> 2. Not force the loading of all project files at startup

If we allow targets in the main file to depend on targets in other
files, we'll have to load and parse this other file anyway, don't we?

> 3. Cutdown on redundent scanning at start that is not needed and
> costly in large projects

Scanning for what?  If you are talking about not searching for cycles
in the dependency graph that we are never going to touch, that is fine
with me.

Stefan

Re: [Patch] Target attribute property expansion

[Peter Donald <do...@apache.org>]

On Wed,  4 Jul 2001 23:48, Stefan Bodewig wrote:
> > For what I saw of the patch, it assumes that properties will be
> > evaluated as they are parsed. Since as you say it preserves DAG
> > being created at parsing time. That would mean that we will be stuck
> > on having to execute <property> directives while we parse, instead
> > of in an initialization phase. Am I wrong in my understanding?
>
> Either that or construct the DAG after that initialization phase
> (where initialization phase would execute all tasks that have not been
> wrapped into targets, if they exist).

I actually think that we should do lazy evaluation of the DAG. ie only look 
up next element in DAG as we "get" to it. This is for two reasons mainly;

1. Support crazy self-modifying projects like Jose wanted
2. Not force the loading of all project files at startup
3. Cutdown on redundent scanning at start that is not needed and costly in 
large projects

Cheers,

Pete

*-----------------------------------------------------*
| "Faced with the choice between changing one's mind, |
| and proving that there is no need to do so - almost |
| everyone gets busy on the proof."                   |
|              - John Kenneth Galbraith               |
*-----------------------------------------------------*

Re: [Patch] Target attribute property expansion

[Stefan Bodewig <bo...@apache.org>]

On Wed, 4 Jul 2001, Jose Alberto Fernandez <j_...@yahoo.com>
wrote:

> I have been trying to stay away from these discussions, since I have
> other work to do, but here I go again. I hope I can keep it short
> and these does not become a major exchange, so here I go.

I'll do my best to not waste your time 8-)

> For what I saw of the patch, it assumes that properties will be
> evaluated as they are parsed. Since as you say it preserves DAG
> being created at parsing time. That would mean that we will be stuck
> on having to execute <property> directives while we parse, instead
> of in an initialization phase. Am I wrong in my understanding?

Either that or construct the DAG after that initialization phase
(where initialization phase would execute all tasks that have not been
wrapped into targets, if they exist).

> Do (or will) <target>s go thru an initialization phase?

Don't know, I see little reason why they should not.

Stefan

RE: [Patch] Target attribute property expansion

[Jose Alberto Fernandez <j_...@yahoo.com>]

> From: Stefan Bodewig [mailto:bodewig@apache.org]
>
> Jose Alberto Fernandez <j_...@yahoo.com> wrote:
>
> > I do not want to be a pain in the neck, but allowing properties in
> > the depends list seems kind of risky to me.
>
> It has not been an unrestricted expansion of properties, but would
> happen at parser time, this wouldn't affect the DAG at all.
>

I have been trying to stay away from these discussions, since I have other
work to do, but here I go again. I hope I can keep it short and these does
not become a major exchange, so here I go.

For what I saw of the patch, it assumes that properties will be evaluated as
they are parsed. Since as you say it preserves DAG being created at parsing
time. That would mean that we will be stuck on having to execute <property>
directives while we parse, instead of in an initialization phase. Am I wrong
in my understanding?

Do (or will) <target>s go thru an initialization phase? I bring this up
because in some of the ANT2 prototypes there was talk about such phase
separate from parsing.

Let me also say, I understand to certain extend the request for being able
to parameterized dependencies, that to me is trying to elevate dependencies
(or tergets) to first class object status. Which is equivalent to allowing
closures (or procedures) as parameters in procedural/functional languages.
One ussually need to be careful of how such things are done, and its
consequences.

Jose Alberto

Re: [Patch] Target attribute property expansion

[Stefan Bodewig <bo...@apache.org>]

Jose Alberto Fernandez <j_...@yahoo.com> wrote:

> I do not want to be a pain in the neck, but allowing properties in
> the depends list seems kind of risky to me.

It has not been an unrestricted expansion of properties, but would
happen at parser time, this wouldn't affect the DAG at all.

Stefan

RE: [Patch] Target attribute property expansion

[Jose Alberto Fernandez <j_...@yahoo.com>]

I do not want to be a pain in the neck, but allowing properties in the
depends list seems kind of risky to me. Now the DAG of dependencies which is
the basis for and evaluation cannot be stablish until one have done property
resolution (at least global properties).

This feels to me like writing gotos to a pointer variable. Good for
assembler but not in structured languages. Do we really want to go that way?

Jose Alberto

> -----Original Message-----
> From: Peter Vogel [mailto:pvogel@arsin.com]
> Sent: Tuesday, July 03, 2001 22:30
> To: 'ant-dev@jakarta.apache.org'
> Subject: [Patch] Target attribute property expansion
>
>
> This is a re-submit based on Stefan's comments.  It expands
> only the depends
> and description attributes,
> and includes some documentation updates as well.
>
> -Peter
>
>  <<targattrs.diff>>
>