scenerios for generating sources

[Jason van Zyl <jv...@maven.org>]

Hi,

I'm in the process of adding facilities for a general approach to
generating sources and I just wanted to field any ideas if anyone has
some input.

I think, from past discussions, we have decided that we will allow
generative plugins to deposit their products to:

${basedir}/target/generated-sources/<plugin-id> (1)

So in my particular case I have:

${basedir}/target/generated-sources/modello (2)

Now, I'm going to take care of getting the information about the
generated directories of sources back to maven's lifecycle using the
analog of the request making process (i.e. using OGNL to extract
response values that can be fed into maven) but I'm wondering if (2) is
fine grained enough.

For the project I'm working on I have blog model and a model for a
particular version of RSS. Now can anyone see why I might not want to
generate all the sources to the same place? I'm just throwing this
question out there as I don't know myself. Would you ever want to
distinguish between various output that is generated by a single plugin
in a project? In this case do I want to distinguish between modello
generated sources for the blog model and the RSS model?

We could always extract an extra parameter within the plugin somehow so
that's not really a problem if we decide to do that. In the case of
modello we could use the id of the model itself. So we could have

${basedir}/target/generated-sources/modello/rss

and

${basedir}/target/generated-sources/modello/blog

Just trying to remain flexible while providing enough control over the
generated sources if you wanted to do something with specific sources
post generation.

-- 
jvz.

Jason van Zyl
jason@maven.org
http://maven.apache.org

happiness is like a butterfly: the more you chase it, the more it will
elude you, but if you turn your attention to other things, it will come
and sit softly on your shoulder ...

 -- Thoreau 

Re: scenerios for generating sources

[John Casey <jd...@commonjava.org>]

> I think it's rather a problem with eclipse being careful about changes in
> directories defined as source directories but not about changes in
> directories itself.
> I can imagine that eclipse team will tell that it's a maven problem
> (because it's
> not a normal situation, that directory disapear) and maven team that it's an
> eclipse problem :)
> And the problem is strictly between both two worlds.
> 


It seems to me that this is a problem with out-of-band updates to your 
project directory, regardless of which side is watching and which side 
if acting. IMO, there are only two ways to solve this problem: 
continuously poll the directory structure, performing incremenal builds 
and/or other adjustments as required; or, integrate the two tools so 
that no changes are out-of-band for either tool...that is, there is a 
notification mechanism that allows the tools to work together and share 
information about stale files, added/removed directories, etc. In other 
words, a fully-integrated plugin for the IDE which runs maven builds. In 
such a plugin, the IDE would inject a build listener and maven would 
notify it via that listener whenever updates to its in-memory model were 
required.

BTW, at my work we used to use an output directory of target/classes, 
and generate via xdoclet to target/generated-sources without a single 
problem. We did this to generate the interfaces and descriptors for 
EJBs, and I've never seen a problem with code generation that 
(Right-Click -> Refresh) wouldn't cure. However, I do know that 
eclipse-3 has gotten much more proactive about noticing this type of 
change...

-john

Re: scenerios for generating sources

[Tomasz Pik <to...@gmail.com>]

On Thu, 30 Sep 2004 20:19:23 +1000, Brett Porter <br...@apache.org> wrote:
> Is this just because you've picked target as a type of output directory
> that it deletes/ignores?

Before moving 'generated' to ${basedir} it was set up in this way:
* src/java - for both eclipse and maven
* target/generated - for both eclipse (second src directory) and maven (addpath)
* target/classes - for both eclipse and maven (so maven do not have to compile
  code if running under eclipse)

And everything works fine till one call maven clean goal.
Afrer clean and new 'generation' round eclipse has problems with refreshing
project and finding new target/generated directory - probably because eclipse
watches target/generated as directory for changes and this handle
points to nothing.

> Maybe attacking the problem and filing an eclipse bug is a good idea :)

I think it's rather a problem with eclipse being careful about changes in
directories defined as source directories but not about changes in
directories itself.
I can imagine that eclipse team will tell that it's a maven problem
(because it's
not a normal situation, that directory disapear) and maven team that it's an
eclipse problem :)
And the problem is strictly between both two worlds.

Regards,
Tomek

> 
> - Brett
> 
> 
> 
> Tomasz Pik wrote:
> 
> >On 29 Sep 2004 12:22:58 -0400, Jason van Zyl <jv...@maven.org> wrote:
> >
> >
> >>Hi,
> >>
> >>I'm in the process of adding facilities for a general approach to
> >>generating sources and I just wanted to field any ideas if anyone has
> >>some input.
> >>
> >>I think, from past discussions, we have decided that we will allow
> >>generative plugins to deposit their products to:
> >>
> >>${basedir}/target/generated-sources/<plugin-id> (1)
> >>
> >>
> >
> >[...]
> >
> >One minor comment.
> >We (at my work) found that having 'generated' classes inside target directory
> >causes some with eclipse and maven clean/regenerate lifecycle - sometimes
> >eclipse goes crazy and has problems with refreshing.
> >So finally we decided to have 'generated' directory right in
> >${basedir}, not in target
> >subdirectory. Just one more entry in .cvsignore.
> >I know it don't play good with maven way of having all temporary
> >things in target
> >subdir but makes our life much more easier.
> >
> >Regards,
> >Tomek
> >
> >PS And I know this directory will be configurable using project property so
> >it's just a note

Re: scenerios for generating sources

[Brett Porter <br...@apache.org>]

Is this just because you've picked target as a type of output directory 
that it deletes/ignores?

Maybe attacking the problem and filing an eclipse bug is a good idea :)

- Brett

Tomasz Pik wrote:

>On 29 Sep 2004 12:22:58 -0400, Jason van Zyl <jv...@maven.org> wrote:
>  
>
>>Hi,
>>
>>I'm in the process of adding facilities for a general approach to
>>generating sources and I just wanted to field any ideas if anyone has
>>some input.
>>
>>I think, from past discussions, we have decided that we will allow
>>generative plugins to deposit their products to:
>>
>>${basedir}/target/generated-sources/<plugin-id> (1)
>>    
>>
>
>[...]
>
>One minor comment.
>We (at my work) found that having 'generated' classes inside target directory
>causes some with eclipse and maven clean/regenerate lifecycle - sometimes
>eclipse goes crazy and has problems with refreshing.
>So finally we decided to have 'generated' directory right in
>${basedir}, not in target
>subdirectory. Just one more entry in .cvsignore.
>I know it don't play good with maven way of having all temporary
>things in target
>subdir but makes our life much more easier.
>
>Regards,
>Tomek
>
>PS And I know this directory will be configurable using project property so
>it's just a note
>
>
>
>  
>

Re: scenerios for generating sources

[Tomasz Pik <to...@gmail.com>]

On 29 Sep 2004 12:22:58 -0400, Jason van Zyl <jv...@maven.org> wrote:
> Hi,
> 
> I'm in the process of adding facilities for a general approach to
> generating sources and I just wanted to field any ideas if anyone has
> some input.
> 
> I think, from past discussions, we have decided that we will allow
> generative plugins to deposit their products to:
> 
> ${basedir}/target/generated-sources/<plugin-id> (1)

[...]

One minor comment.
We (at my work) found that having 'generated' classes inside target directory
causes some with eclipse and maven clean/regenerate lifecycle - sometimes
eclipse goes crazy and has problems with refreshing.
So finally we decided to have 'generated' directory right in
${basedir}, not in target
subdirectory. Just one more entry in .cvsignore.
I know it don't play good with maven way of having all temporary
things in target
subdir but makes our life much more easier.

Regards,
Tomek

PS And I know this directory will be configurable using project property so
it's just a note

Re: scenerios for generating sources

[Brett Porter <br...@apache.org>]

Quoting Jason van Zyl <jv...@maven.org>:

> On Wed, 2004-09-29 at 16:52, Brett Porter wrote:
> > If this is the case, I think generate to the same directory, and if 
> > that's a problem for you, create subprojects. I also think Michal has a 
> > good point about using the standard layout.
> 
> I only want a standard layout, what I was saying is whether it should be
> parameterized. But I agree with Emmanuel in that it is best to make a
> small project which generates the sources as part of a larger project.
> If there are no counter examples as to why this can't work in practice
> then that's what I would like to promote. A large project may have
> several models, say, but each of those can go into a separate build.
> That's is clear, and definitely scalable.
> 

Whn I was referring to standard layout, I meant
target/generated-sources/foo/java, target/generated-sources/foo/resources, etc.

I think it can be up to the plugin - I would think modello should be able to
take several related mdo files and generate one source tree. But one plugin, one
source tree definitely sounds right - if you need to actually separate the
generated sources, create subprojects.

- Brett

Re: scenerios for generating sources

[Jason van Zyl <jv...@maven.org>]

On Wed, 2004-09-29 at 16:52, Brett Porter wrote:
> If this is the case, I think generate to the same directory, and if 
> that's a problem for you, create subprojects. I also think Michal has a 
> good point about using the standard layout.

I only want a standard layout, what I was saying is whether it should be
parameterized. But I agree with Emmanuel in that it is best to make a
small project which generates the sources as part of a larger project.
If there are no counter examples as to why this can't work in practice
then that's what I would like to promote. A large project may have
several models, say, but each of those can go into a separate build.
That's is clear, and definitely scalable.

-- 
jvz.

Jason van Zyl
jason@maven.org
http://maven.apache.org

happiness is like a butterfly: the more you chase it, the more it will
elude you, but if you turn your attention to other things, it will come
and sit softly on your shoulder ...

 -- Thoreau 

Re: scenerios for generating sources

[Brett Porter <br...@apache.org>]

If this is the case, I think generate to the same directory, and if 
that's a problem for you, create subprojects. I also think Michal has a 
good point about using the standard layout.

I wonder though, is it possible that the plugin could decide using 
guidelines? So if a plugin needed to create several sets of generated 
sources, it could pick the locations and still have Maven pick them up? 
I think this actually works well with m1, we just didn't have the 
consistency, nor a good practice for using the compile.src.set in other 
plugins.

That's an important aspect here: to make sure that plugins never use 
project.build.sourceDirectory (unless that's what they really truly 
need), but use the full list of source paths.

Just on another note, with Michal's example about middlegen: does that 
plugin really do both the generation and xdoclet? I would think that it 
would be :
- middlegen generates sources to its directory
- xdoclet takes those sources and generates its own and is a separate 
plugin.

One final thing: what about transforming sources? I'm sure there is a 
use case (though I can't think of a product that does it), for being 
able to generate sources and remove the original from the source path, 
effectively transforming one source set to another.

Cheers,
Brett

Jason van Zyl wrote:

>On Wed, 2004-09-29 at 13:00, Emmanuel Venisse wrote:
>
>  
>
>>Why you generate sources in the same projects? If this model are
>>independant, we can generate them in independant projects.
>>    
>>
>
>I suppose I could, there is no reason I couldn't. The models are
>definitely independent but they contribute to an application model as a
>whole. Dealing with them separately would definitely make things easier
>and I agree that would be better.
>
>Can you think of any counter examples that would make doing somethings
>extremely difficult or impossible. Modello is the example here but what
>about other tools.
>
>I'm in agreement with you, if this could work for everything I am happy
>to just put a model in its own project and generate it there. It would
>definitely make things clear.
>
>  
>
>>Emmanuel
>>    
>>
>
>  
>

Re: scenerios for generating sources

[Jason van Zyl <jv...@maven.org>]

On Wed, 2004-09-29 at 13:00, Emmanuel Venisse wrote:

> 
> Why you generate sources in the same projects? If this model are
> independant, we can generate them in independant projects.

I suppose I could, there is no reason I couldn't. The models are
definitely independent but they contribute to an application model as a
whole. Dealing with them separately would definitely make things easier
and I agree that would be better.

Can you think of any counter examples that would make doing somethings
extremely difficult or impossible. Modello is the example here but what
about other tools.

I'm in agreement with you, if this could work for everything I am happy
to just put a model in its own project and generate it there. It would
definitely make things clear.

> Emmanuel

-- 
jvz.

Jason van Zyl
jason@maven.org
http://maven.apache.org

happiness is like a butterfly: the more you chase it, the more it will
elude you, but if you turn your attention to other things, it will come
and sit softly on your shoulder ...

 -- Thoreau 

Re: scenerios for generating sources

[Emmanuel Venisse <em...@venisse.net>]

----- Original Message ----- 
From: "Jason van Zyl" <jv...@maven.org>
To: "Maven 2 Developers List" <m2...@maven.apache.org>
Sent: Wednesday, September 29, 2004 6:22 PM
Subject: scenerios for generating sources


> Hi,
>
> I'm in the process of adding facilities for a general approach to
> generating sources and I just wanted to field any ideas if anyone has
> some input.
>
> I think, from past discussions, we have decided that we will allow
> generative plugins to deposit their products to:
>
> ${basedir}/target/generated-sources/<plugin-id> (1)
>
> So in my particular case I have:
>
> ${basedir}/target/generated-sources/modello (2)
>
> Now, I'm going to take care of getting the information about the
> generated directories of sources back to maven's lifecycle using the
> analog of the request making process (i.e. using OGNL to extract
> response values that can be fed into maven) but I'm wondering if (2) is
> fine grained enough.
>
> For the project I'm working on I have blog model and a model for a
> particular version of RSS. Now can anyone see why I might not want to
> generate all the sources to the same place? I'm just throwing this
> question out there as I don't know myself. Would you ever want to
> distinguish between various output that is generated by a single plugin
> in a project? In this case do I want to distinguish between modello
> generated sources for the blog model and the RSS model?
>
> We could always extract an extra parameter within the plugin somehow so
> that's not really a problem if we decide to do that. In the case of
> modello we could use the id of the model itself. So we could have
>
> ${basedir}/target/generated-sources/modello/rss
>
> and
>
> ${basedir}/target/generated-sources/modello/blog
>
> Just trying to remain flexible while providing enough control over the
> generated sources if you wanted to do something with specific sources
> post generation.
>

Why you generate sources in the same projects? If this model are
independant, we can generate them in independant projects.

Emmanuel