You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Thomas Broyer <t....@gmail.com> on 2012/12/17 11:50:44 UTC
Seeking feedback on “Recursive Maven considered harmful”
Hi,
On his blog, Lex Spoon makes some interesting points against most
build systems, using Maven as an example:
http://blog.lexspoon.org/2012/12/recursive-maven-considered-harmful.html
In related news, I've heard people complain about “insanely long build
times via building unnecessary things” and tracking down the cause to
being Maven, to the extent that they're thinking about making their
own build system (many are Xooglers and they probably would like.
I've made some proposals for improvements in the comments of Lex's
post, and I'm eager to know what other people think, particularly
Maven core developers/contributors. Has this been ever discussed? If
yes, is this planned or has it been dismissed? In the latter case, for
which reasons?
Thanks in advance,
--
Thomas Broyer
/tɔ.ma.bʁwa.je/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Seeking feedback on “Recursive Maven considered harmful”
Posted by Stephen Connolly <st...@gmail.com>.
Reformatted and edited into something a bit more specific:
http://developer-blog.cloudbees.com/2012/12/maven-and-hack.html
On 17 December 2012 12:45, Stephen Connolly <stephen.alan.connolly@gmail.com
> wrote:
> And I try to remove such bugs as and when I find them... but yes I agree
> it's a pain... but people should be more aware that it is a hack and they
> would be better served by fixing the root cause... not applying the
> "install" hack
>
>
> On 17 December 2012 12:26, Benson Margulies <bi...@gmail.com> wrote:
>
>> with respect to 'the install hack': If I had a dollar for every
>> occasion where a bug in a plugin or the core required me to use it,
>> I'd be a richer man by a considerable amount.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>>
>>
>
Re: Seeking feedback on “Recursive Maven considered harmful”
Posted by Thomas Broyer <t....@gmail.com>.
On Mon, Dec 17, 2012 at 2:08 PM, Arnaud Héritier <ah...@gmail.com> wrote:
> But yes all plugins should have an update/incremental behavior but it's not
> easy to do because there are many factors that may require to rebuild some
> parts of your project (You may edit your settings.xml, a pom.xml ....).
> It's not (sadly) as simple as one source file -> one compiled file because
> of all things possible with java (inner-classes, code generation from
> annotations ...) or with some additional libs. The problem is that an
> incremental build that mostly work may be worst than no incremental build
> at all because instead of loosing always many time to have a secured result
> you can regularly lost a lot of time because of a random behavior.
There are bugs in every piece of software. Put the new incremental
behavior behind a flag to make it easy to revert to the old (current)
behavior and should be done. Most plugins already try to detect when
they need to do do work, so it's more an evolution (that can be
disable-able) than a revolution/rewrite.
> Otherwise I'm agree with the blog post and would dream to have something as
> performant as what we can have with make.
Let's all move back to Make! :-P
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Seeking feedback on “Recursive Maven considered harmful”
Posted by Arnaud Héritier <ah...@gmail.com>.
The problem is also with Java itself.
Let's take a project with a war depending of a jar produced by another
module or an ear relying on several war and you cannot use anymore
something below install (it should be package to at least find the archive
in the target dir and not to have to find it in the local repo)
But yes all plugins should have an update/incremental behavior but it's not
easy to do because there are many factors that may require to rebuild some
parts of your project (You may edit your settings.xml, a pom.xml ....).
It's not (sadly) as simple as one source file -> one compiled file because
of all things possible with java (inner-classes, code generation from
annotations ...) or with some additional libs. The problem is that an
incremental build that mostly work may be worst than no incremental build
at all because instead of loosing always many time to have a secured result
you can regularly lost a lot of time because of a random behavior.
Otherwise I'm agree with the blog post and would dream to have something as
performant as what we can have with make.
Arnaud
On Mon, Dec 17, 2012 at 1:45 PM, Stephen Connolly <
stephen.alan.connolly@gmail.com> wrote:
> And I try to remove such bugs as and when I find them... but yes I agree
> it's a pain... but people should be more aware that it is a hack and they
> would be better served by fixing the root cause... not applying the
> "install" hack
>
>
> On 17 December 2012 12:26, Benson Margulies <bi...@gmail.com> wrote:
>
> > with respect to 'the install hack': If I had a dollar for every
> > occasion where a bug in a plugin or the core required me to use it,
> > I'd be a richer man by a considerable amount.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> > For additional commands, e-mail: users-help@maven.apache.org
> >
> >
>
Re: Seeking feedback on “Recursive Maven considered harmful”
Posted by Stephen Connolly <st...@gmail.com>.
And I try to remove such bugs as and when I find them... but yes I agree
it's a pain... but people should be more aware that it is a hack and they
would be better served by fixing the root cause... not applying the
"install" hack
On 17 December 2012 12:26, Benson Margulies <bi...@gmail.com> wrote:
> with respect to 'the install hack': If I had a dollar for every
> occasion where a bug in a plugin or the core required me to use it,
> I'd be a richer man by a considerable amount.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
Re: Seeking feedback on “Recursive Maven considered harmful”
Posted by Benson Margulies <bi...@gmail.com>.
with respect to 'the install hack': If I had a dollar for every
occasion where a bug in a plugin or the core required me to use it,
I'd be a richer man by a considerable amount.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Seeking feedback on “Recursive Maven considered harmful”
Posted by Thomas Broyer <t....@gmail.com>.
On Mon, Dec 17, 2012 at 12:36 PM, Stephen Connolly
<st...@gmail.com> wrote:
> Current work on Maven compiler plugin version 3.0 is adding support for an
> incremental mode.
>
> It is a tricky balance to cut, as if incremental mode is too aggressive,
> you compile more than is strictly necessary... and if too lax, you compile
> less than is required and people end up having to run "clean" first again.
>
> Another issue is that people abuse the "install" hack far more often than
> they should...
>
> In fact it seems most people's default mode with Maven is to go "mvn clean
> install".
>
> There are a lot of issues with that, not least being that you are abusing
> the local repository cache.
I'm so glad you mention this, I though I was alone in thinking that way!
See for example the responses to
http://stackoverflow.com/q/10686682/116472 and the total lack of reply
to http://users.markmail.org/thread/3tpgtmsisxbdccw2
This should be more prominently explained in the docs though.
And I believe one reason people do that, is that Maven is not really
good at making "not install" a sensible default: "-rf" in a reactor
build will resolves skipped modules from the repository!
> Where this really kicks in is when people have made their build to *rely*
> on artifacts being installed in the cache... cue ensuing hilarity when
> using the release plugin... "solved" (if even detected) by people changing
> to <preparationGoals>clean install</preparationGoals>... of course the real
> solution is to fix your build so that it works with "clean verify" on a
> 'virgin' set of version numbers.
>
> One of your points about partial phases actually misses one of the key
> things about Maven that a lot of people have missed... namely that you
> don't have to go all the way.
>
> The extension of my "install" hack complaint is the "package" hack
> complaint.
>
> If your build requires you to go as far as the "package" phase, then likely
> you are missing out on doing it right... might not be your fault, may the
> the fault of some plugins you are using... but none the less there are
> things you are missing out on.
>
> My hope is that, for example, some of the tricks I am applying with the
> jszip.org's maven plugin will enlighten people into the benefits of getting
> your build 'right'... for example
>
> $ mvn compile jszip:run
>
> In a multi-module build... now your java classpath is not for the
> **/target/*.jar files but for the **/target/classes directories... so you
> can have your IDE compile the changed classes, and the classpath scanning
> of jszip:run will pick up the changes and reload the servlet container...
> With the background compile feature in for example IDEA 12 (I ack that
> eclipse's webby plugin does similar to jszip:run... some of my tricks were
> picked up from Benjamin) you basically can just write your java and
> javascript code and switch back to the browser, hit reload and see the
> changes...
Nit: you might want to use "mvn process-classes jszip:run" instead of
"compile" ;-)
> If you do "mvn package jszip:run" however the `target/classes` directory
> will have been replaced as the primary artifact by the
> `target/${finalName}.jar` and therefore we are limited to watching for the
> .jar file to be modified... no more live development. Yes I could hack and
> infer the target/classes directory... but that hack would not be the Maven
> way.
It unfortunately basically only works for the "main artifact" of a
packaging=jar module. AFAICT, it won't work if you need the
java-source or test-jar (fixed for test-jars in the upcoming Maven
3.1: http://jira.codehaus.org/browse/MNG-5214, but for java-source you
have to code your plugins to use getReferenceProjects() and
getCompileSourceRoots() –getTestCompileSourceRoots if the dependency
has classifier=test-sources– when Artifact#getFile() is null) I'm
ready to admit depending on java-source artifacts is kind of an
edge-case though (even though it's more than current practice with
GWT).
> So the phase you invoke depends on your needs.
>
> You are doing Maven wrong (IMHO) if your entire build cannot work under the
> following incremental behaviours:
>
> 1. Change all the modules to a 'virgin' version number (one that has never
> been touched before... don't want the local repo or any remote repos
> "helping" us out
> 2. "mvn clean" if you cannot clean a clean fresh checkout on virgin version
> numbers #fail
> 3. "mvn clean generate-sources" if you cannot create all the generated
> source code, your IDE experience will be less than stellar #fail
This can fail in reactor builds, when generating sources for a module
requires compiled classes from a dependency module.
One answer to that is that the choice between "process-classes" and
"package" in the multi-module scenario above shouldn't be left at the
developer's appreciation. More often than not, what I want to do is
not "run this goal (or up to this phase) on all modules" but rather
"run this goal (or up to this phase) on this submodule (and possibly
the modules that depends on it), and simply package (or
compile/process-classes) on the modules it depends on". Currently, I'd
have to "mvn install" a few modules to be able to run "mvn some-goal
-rf moduleX". What I generally want is "build everything needed to do
X", and I do not really care whether "build" means "package" or
"process-classes" (and as Arnaud said, it's also dependent on the
packaging types).
But note that Peter Miller's view is to rather always build
"everything" ("mvn package" or "mvn verify"): compile and run tests
everywhere and let the build tool decide what it really *needs* to do.
Maven's current situation is a mix of "build too few" (compilation,
leading to "mvn clean"), "build too much" (running tests; leading to
"mvn install" and "mvn -rf X"), and other side-effects of not tracking
changes (compilation, resource-copying and packaging; leading again to
"mvn clean"); nothing that cannot be fixed though.
--
Thomas Broyer
/tɔ.ma.bʁwa.je/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Seeking feedback on “Recursive Maven considered harmful”
Posted by Mark Struberg <st...@yahoo.de>.
how do you know the resource handling is not depending on other jars?
I know this is not perfect, but if you have a bigger project then you have a tree of depending modules. And not every module is depending on any other. So while a forced 'clean' on _those_ depending modules is more expensive then just doing nothing it is still the only practical way. Otherwise the plugins would need to know exactly what happened in the other modules.
Please also note that we do treat external dependencies pretty similar to reactor modules once we reach a certain phase in the build lifecycle.
I know that doing this 'clean' is not free of cost, but this is still _way_ cheaper than doing a mvn clean install in any case as it is needed right now!
LieGrue,
strub
----- Original Message -----
> From: Thomas Broyer <t....@gmail.com>
> To: Maven Users List <us...@maven.apache.org>; Mark Struberg <st...@yahoo.de>
> Cc:
> Sent: Monday, December 17, 2012 3:16 PM
> Subject: Re: Seeking feedback on “Recursive Maven considered harmful”
>
> On Mon, Dec 17, 2012 at 12:47 PM, Mark Struberg <st...@yahoo.de> wrote:
>> I really like to pick up the work again, but currently busy with graduating
> another project.
>>
>> The next important points to do are
>>
>> * inter-project change detection. If you have a dependency to another
> project which changed, you need to basically do a full build on your depending
> module. Thus said, if we have this we do _not_ need to rebuild any other modules
> which do not have such a dependency.
>>
>> * making maven-resource-plugin incremential ready. Later on other plugins,
> but m-r-p is really used a lots and if you have a new config, then you most
> likely also like to rebuild/retest your project
>>
>> * surefire integration. If no dependency got changed and no code/previous
> stuff got changed, you most probably also do not like to run all the tests
> again. This could _seriously_ cut down build times!
>
> DISCLAIMER: I'm over-simplifying things here by only considering Java
> projects, I know it's more complex than that.
>
> Some of the things above above and in
> https://cwiki.apache.org/MAVEN/incremental-builds.html are kind of
> scary!
> Because one of your dependencies has changed does not mean you have to
> do a "clean": let each plugin decide. For instance, the resources
> plugin doesn't really care what your dependencies are. The compiler
> plugin will use some of them though, and it can simply include them in
> its staleness check.
>
> Ideally the compiler plugin would track dependencies at the class (or
> file) level. In the example from the wiki, it would store in
> target/maven.status/compiler.status that BeanA2 depends on BeanA and
> thus needs to be recompiled if BeanA has changed. In ModuleB, it could
> be enough to track that BeanA comes from ModuleA and that BeanB thus
> needs to be recompiled if the ModuleA dependency has changed. Because
> we're in a multi-module build though, we might want to track the
> direct dependency to BeanA as ModuleB could be compiled with the
> target/classes of ModuleA, so you could skip recompiling BeanB if only
> BeanA2 has changed. AIUI, IDEs are already doing such dependency
> tracking, and it does not seem to be taking that much time to compute
> the dependency graph and then incrementally update it as files change.
> The compiler plugin also needs to remove BeanA2.class from
> target/classes when I delete BeanA2.java from src/main/java or I
> rename it (and similarly for the resources plugin).
>
> Similarly, you'd want to track class/files dependencies to only run
> those tests related to the classes that changed. A less ambitious goal
> would, as you propose, only skip running tests altogether when
> nothing's changed, and run all of them otherwise (as today), and that
> would already save all of us a huge amount of build time.
>
> This is not an easy task though, I know it, I know it cannot be done
> in a matter of days or weeks, but it looks to me like it can be done
> incrementally, one plugin at a time (the wiki also discusses
> properties and profiles, which I hadn't thought about, but it seems to
> me like this is not that different: take the plugin's configuration
> into account when computing the work that needs to be done; the only
> change needed at the core level would be tracking which plugins ran in
> a previous build that won't run in the current one, it could then call
> the plugin's "clean" goal if it exists at the beginning of the
> build,
> with its previous configuration, and otherwise clean the whole module)
>
> I might very well miss something obvious here though, as I'm merely
> thinking out loud. Feel free to disregard my statements or contradict
> me, I'm here to learn.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Seeking feedback on “Recursive Maven considered harmful”
Posted by Thomas Broyer <t....@gmail.com>.
On Mon, Dec 17, 2012 at 12:47 PM, Mark Struberg <st...@yahoo.de> wrote:
> I really like to pick up the work again, but currently busy with graduating another project.
>
> The next important points to do are
>
> * inter-project change detection. If you have a dependency to another project which changed, you need to basically do a full build on your depending module. Thus said, if we have this we do _not_ need to rebuild any other modules which do not have such a dependency.
>
> * making maven-resource-plugin incremential ready. Later on other plugins, but m-r-p is really used a lots and if you have a new config, then you most likely also like to rebuild/retest your project
>
> * surefire integration. If no dependency got changed and no code/previous stuff got changed, you most probably also do not like to run all the tests again. This could _seriously_ cut down build times!
DISCLAIMER: I'm over-simplifying things here by only considering Java
projects, I know it's more complex than that.
Some of the things above above and in
https://cwiki.apache.org/MAVEN/incremental-builds.html are kind of
scary!
Because one of your dependencies has changed does not mean you have to
do a "clean": let each plugin decide. For instance, the resources
plugin doesn't really care what your dependencies are. The compiler
plugin will use some of them though, and it can simply include them in
its staleness check.
Ideally the compiler plugin would track dependencies at the class (or
file) level. In the example from the wiki, it would store in
target/maven.status/compiler.status that BeanA2 depends on BeanA and
thus needs to be recompiled if BeanA has changed. In ModuleB, it could
be enough to track that BeanA comes from ModuleA and that BeanB thus
needs to be recompiled if the ModuleA dependency has changed. Because
we're in a multi-module build though, we might want to track the
direct dependency to BeanA as ModuleB could be compiled with the
target/classes of ModuleA, so you could skip recompiling BeanB if only
BeanA2 has changed. AIUI, IDEs are already doing such dependency
tracking, and it does not seem to be taking that much time to compute
the dependency graph and then incrementally update it as files change.
The compiler plugin also needs to remove BeanA2.class from
target/classes when I delete BeanA2.java from src/main/java or I
rename it (and similarly for the resources plugin).
Similarly, you'd want to track class/files dependencies to only run
those tests related to the classes that changed. A less ambitious goal
would, as you propose, only skip running tests altogether when
nothing's changed, and run all of them otherwise (as today), and that
would already save all of us a huge amount of build time.
This is not an easy task though, I know it, I know it cannot be done
in a matter of days or weeks, but it looks to me like it can be done
incrementally, one plugin at a time (the wiki also discusses
properties and profiles, which I hadn't thought about, but it seems to
me like this is not that different: take the plugin's configuration
into account when computing the work that needs to be done; the only
change needed at the core level would be tracking which plugins ran in
a previous build that won't run in the current one, it could then call
the plugin's "clean" goal if it exists at the beginning of the build,
with its previous configuration, and otherwise clean the whole module)
I might very well miss something obvious here though, as I'm merely
thinking out loud. Feel free to disregard my statements or contradict
me, I'm here to learn.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Seeking feedback on “Recursive Maven considered harmful”
Posted by Mark Struberg <st...@yahoo.de>.
I really like to pick up the work again, but currently busy with graduating another project.
The next important points to do are
* inter-project change detection. If you have a dependency to another project which changed, you need to basically do a full build on your depending module. Thus said, if we have this we do _not_ need to rebuild any other modules which do not have such a dependency.
* making maven-resource-plugin incremential ready. Later on other plugins, but m-r-p is really used a lots and if you have a new config, then you most likely also like to rebuild/retest your project
* surefire integration. If no dependency got changed and no code/previous stuff got changed, you most probably also do not like to run all the tests again. This could _seriously_ cut down build times!
LieGrue,
strub
----- Original Message -----
> From: Stephen Connolly <st...@gmail.com>
> To: Maven Users List <us...@maven.apache.org>
> Cc:
> Sent: Monday, December 17, 2012 12:36 PM
> Subject: Re: Seeking feedback on “Recursive Maven considered harmful”
>
> Current work on Maven compiler plugin version 3.0 is adding support for an
> incremental mode.
>
> It is a tricky balance to cut, as if incremental mode is too aggressive,
> you compile more than is strictly necessary... and if too lax, you compile
> less than is required and people end up having to run "clean" first
> again.
>
> Another issue is that people abuse the "install" hack far more often
> than
> they should...
>
> In fact it seems most people's default mode with Maven is to go "mvn
> clean
> install".
>
> There are a lot of issues with that, not least being that you are abusing
> the local repository cache.
>
> Where this really kicks in is when people have made their build to *rely*
> on artifacts being installed in the cache... cue ensuing hilarity when
> using the release plugin... "solved" (if even detected) by people
> changing
> to <preparationGoals>clean install</preparationGoals>... of course
> the real
> solution is to fix your build so that it works with "clean verify" on
> a
> 'virgin' set of version numbers.
>
> One of your points about partial phases actually misses one of the key
> things about Maven that a lot of people have missed... namely that you
> don't have to go all the way.
>
> The extension of my "install" hack complaint is the
> "package" hack
> complaint.
>
> If your build requires you to go as far as the "package" phase, then
> likely
> you are missing out on doing it right... might not be your fault, may the
> the fault of some plugins you are using... but none the less there are
> things you are missing out on.
>
> My hope is that, for example, some of the tricks I am applying with the
> jszip.org's maven plugin will enlighten people into the benefits of getting
> your build 'right'... for example
>
> $ mvn compile jszip:run
>
> In a multi-module build... now your java classpath is not for the
> **/target/*.jar files but for the **/target/classes directories... so you
> can have your IDE compile the changed classes, and the classpath scanning
> of jszip:run will pick up the changes and reload the servlet container...
> With the background compile feature in for example IDEA 12 (I ack that
> eclipse's webby plugin does similar to jszip:run... some of my tricks were
> picked up from Benjamin) you basically can just write your java and
> javascript code and switch back to the browser, hit reload and see the
> changes...
>
> If you do "mvn package jszip:run" however the `target/classes`
> directory
> will have been replaced as the primary artifact by the
> `target/${finalName}.jar` and therefore we are limited to watching for the
> .jar file to be modified... no more live development. Yes I could hack and
> infer the target/classes directory... but that hack would not be the Maven
> way.
>
> So the phase you invoke depends on your needs.
>
> You are doing Maven wrong (IMHO) if your entire build cannot work under the
> following incremental behaviours:
>
> 1. Change all the modules to a 'virgin' version number (one that has
> never
> been touched before... don't want the local repo or any remote repos
> "helping" us out
> 2. "mvn clean" if you cannot clean a clean fresh checkout on virgin
> version
> numbers #fail
> 3. "mvn clean generate-sources" if you cannot create all the generated
> source code, your IDE experience will be less than stellar #fail
> 4. "mvn clean compile" if you cannot compile all your production code
> without bundling up jars and stuffing them in the local repo cache, shame
> on you #fail
> 5. "mvn clean test-compile" if you cannot compile all your test code
> #fail
> 6. "mvn clean package -DskipTests" if those packaged artifacts need to
> go
> into the local repo cache for subsequent modules to find them, you have a
> subtly broken build #fail
> 7. "mvn clean verify -DskipTests" if you cannot use the release
> plugin's
> default test of a releaseable artifact on virgin version numbers: #fail
>
> Note that all these tests are designed to be quick (i.e. not looking at
> whether you have working test cases and ignoring the results of tests) and
> to be run in succession on the same virgin version number(s)
>
> Not that you will use all of the above commands on a regular basis... but
> if you cannot pass all 6 tests, you are not enabling the full power of
> Maven and I personally will not listen to you complaining about how "Maven
> sucks" because the truth is *your* build sucks and therefore we cannot make
> any inference as to the sucky-ness of Maven (which may indeed suck... but I
> have yet to see a build that passes my tests and shows Maven to suck)
>
> -Stephen
>
>
> On 17 December 2012 10:50, Thomas Broyer <t....@gmail.com> wrote:
>
>> Hi,
>>
>> On his blog, Lex Spoon makes some interesting points against most
>> build systems, using Maven as an example:
>> http://blog.lexspoon.org/2012/12/recursive-maven-considered-harmful.html
>> In related news, I've heard people complain about “insanely long build
>> times via building unnecessary things” and tracking down the cause to
>> being Maven, to the extent that they're thinking about making their
>> own build system (many are Xooglers and they probably would like.
>>
>> I've made some proposals for improvements in the comments of Lex's
>> post, and I'm eager to know what other people think, particularly
>> Maven core developers/contributors. Has this been ever discussed? If
>> yes, is this planned or has it been dismissed? In the latter case, for
>> which reasons?
>>
>> Thanks in advance,
>>
>> --
>> Thomas Broyer
>> /tɔ.ma.bʁwa.je/ <http://xn--nna.ma.xn--bwa-xxb.je/>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Seeking feedback on “Recursive Maven considered harmful”
Posted by Mark Struberg <st...@yahoo.de>.
By default it should be fine to use the 'mvn verify' build lifecycle step [1].
In that case the 'reactor' does not reference the project dependencies via the local maven repo but instead via path JVM references to the other modules in the build directly.
LieGrue,
strub
[1] http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html
----- Original Message -----
> From: Laird Nelson <lj...@gmail.com>
> To: Maven Users List <us...@maven.apache.org>
> Cc:
> Sent: Monday, December 17, 2012 4:34 PM
> Subject: Re: Seeking feedback on “Recursive Maven considered harmful”
>
> On Mon, Dec 17, 2012 at 3:36 AM, Stephen Connolly <
> stephen.alan.connolly@gmail.com> wrote:
>
>> Another issue is that people abuse the "install" hack far more
> often than
>> they should...
>>
>
> Hi; this is the first I've read anywhere about the "install
> hack". Where
> can I find out more about it?
>
>
>> In fact it seems most people's default mode with Maven is to go
> "mvn clean
>> install".
>>
>
> Guilty as charged in a multi-module build. Are you saying there's another
> way that modules in a multi-module build can share artifacts? Could you
> point me to where this is documented?
>
>
>> of course the real
>> solution is to fix your build so that it works with "clean
> verify" on a
>> 'virgin' set of version numbers.
>>
>
> To be clear: in a multi-module build, with a root pom, with brand new
> version numbers, you're saying the whole build should run just fine when
> invoked from the root with "clean verify"?
>
> Just another user trying to do things right,
> Best,
> Laird
>
>>
> --
> http://about.me/lairdnelson
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re:Seeking feedback on “Recursive Maven considered harmful”
Posted by Robert Scholte <rf...@apache.org>.
Last month I head the idea to only show the -rf option if you ran Maven
with at least the install goal. The next step was to fix this, either with
serialization of the MavenProjects or xml-file which could be picked when
continuing the build with the -rf argument. Hence, why not work on that
second part immediately, which would make the first step unnecessary.
Right now it's still an idea, haven't worked on it yet.
Also: don't underestimate the power of the -amd flag.
Robert
Op Mon, 17 Dec 2012 17:22:21 +0100 schreef Daniel Kulp <dk...@apache.org>:
>
> On Dec 17, 2012, at 11:02 AM, Stephen Connolly
> <st...@gmail.com> wrote:
>
>> On 17 December 2012 15:34, Laird Nelson <lj...@gmail.com> wrote:
>> I am not saying that you only ever run it with "clean verify"
>>
>> I am not saying running with "clean install" is considered harmful.
>>
>> I am saying that *if* you _need_ to use "clean install" then your build
>> is
>> not the best it could be and you may have issues with your build that
>> you
>> haven't noticed.
>
>
> To be fair, a lot of the "install hack" stuff I've seen is more
> historical and related to issues in Maven <=2.2.1.
>
> For example:
>
> build a few modules that result in some jars/classes/whatever
> build a maven plugin that uses those modules
> use that plugin in later modules
>
> That is likely one of the most common "install hack" cases as that never
> seemed to work correctly (or at all) without actually installing the
> jars and plugins. Honestly, until you mentioned it today, I hadn't
> even tested this case with 3.0.4 to see if it's now fixed. Seems like
> it is. Time to go update builds. :-)
>
>
> Dan
>
>
>
>> If you have a big multi-module build and are only working in one small
>> part, by all means do
>>
>> $ git pull
>> $ mvn clean install
>> $ mvn clean install -pl foo:bar -am (or -amd depending on your needs)
>> $ mvn clean install -pl foo:bar -am
>> ...
>> $ mvn clean install -pl foo:bar -am
>>
>> Just don't assume that you always have to run as far as install... and
>> make
>> sure your build doesn't require it also
>>
>>
>>>
>>> Just another user trying to do things right,
>>> Best,
>>> Laird
>>>
>>>>
>>> --
>>> http://about.me/lairdnelson
>>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Seeking feedback on “Recursive Maven considered harmful”
Posted by Daniel Kulp <dk...@apache.org>.
On Dec 17, 2012, at 11:02 AM, Stephen Connolly <st...@gmail.com> wrote:
> On 17 December 2012 15:34, Laird Nelson <lj...@gmail.com> wrote:
> I am not saying that you only ever run it with "clean verify"
>
> I am not saying running with "clean install" is considered harmful.
>
> I am saying that *if* you _need_ to use "clean install" then your build is
> not the best it could be and you may have issues with your build that you
> haven't noticed.
To be fair, a lot of the "install hack" stuff I've seen is more historical and related to issues in Maven <=2.2.1.
For example:
build a few modules that result in some jars/classes/whatever
build a maven plugin that uses those modules
use that plugin in later modules
That is likely one of the most common "install hack" cases as that never seemed to work correctly (or at all) without actually installing the jars and plugins. Honestly, until you mentioned it today, I hadn't even tested this case with 3.0.4 to see if it's now fixed. Seems like it is. Time to go update builds. :-)
Dan
> If you have a big multi-module build and are only working in one small
> part, by all means do
>
> $ git pull
> $ mvn clean install
> $ mvn clean install -pl foo:bar -am (or -amd depending on your needs)
> $ mvn clean install -pl foo:bar -am
> ...
> $ mvn clean install -pl foo:bar -am
>
> Just don't assume that you always have to run as far as install... and make
> sure your build doesn't require it also
>
>
>>
>> Just another user trying to do things right,
>> Best,
>> Laird
>>
>>>
>> --
>> http://about.me/lairdnelson
>>
--
Daniel Kulp
dkulp@apache.org - http://dankulp.com/blog
Talend Community Coder - http://coders.talend.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Seeking feedback on “Recursive Maven considered harmful”
Posted by Stephen Connolly <st...@gmail.com>.
On 17 December 2012 15:34, Laird Nelson <lj...@gmail.com> wrote:
> On Mon, Dec 17, 2012 at 3:36 AM, Stephen Connolly <
> stephen.alan.connolly@gmail.com> wrote:
>
> > Another issue is that people abuse the "install" hack far more often than
> > they should...
> >
>
> Hi; this is the first I've read anywhere about the "install hack". Where
> can I find out more about it?
>
>
> > In fact it seems most people's default mode with Maven is to go "mvn
> clean
> > install".
> >
>
> Guilty as charged in a multi-module build. Are you saying there's another
> way that modules in a multi-module build can share artifacts? Could you
> point me to where this is documented?
>
>
> > of course the real
> > solution is to fix your build so that it works with "clean verify" on a
> > 'virgin' set of version numbers.
> >
>
> To be clear: in a multi-module build, with a root pom, with brand new
> version numbers, you're saying the whole build should run just fine when
> invoked from the root with "clean verify"?
>
I am not saying that you only ever run it with "clean verify"
I am not saying running with "clean install" is considered harmful.
I am saying that *if* you _need_ to use "clean install" then your build is
not the best it could be and you may have issues with your build that you
haven't noticed.
If you have a big multi-module build and are only working in one small
part, by all means do
$ git pull
$ mvn clean install
$ mvn clean install -pl foo:bar -am (or -amd depending on your needs)
$ mvn clean install -pl foo:bar -am
...
$ mvn clean install -pl foo:bar -am
Just don't assume that you always have to run as far as install... and make
sure your build doesn't require it also
>
> Just another user trying to do things right,
> Best,
> Laird
>
> >
> --
> http://about.me/lairdnelson
>
Re: Seeking feedback on “Recursive Maven considered harmful”
Posted by Laird Nelson <lj...@gmail.com>.
On Mon, Dec 17, 2012 at 3:36 AM, Stephen Connolly <
stephen.alan.connolly@gmail.com> wrote:
> Another issue is that people abuse the "install" hack far more often than
> they should...
>
Hi; this is the first I've read anywhere about the "install hack". Where
can I find out more about it?
> In fact it seems most people's default mode with Maven is to go "mvn clean
> install".
>
Guilty as charged in a multi-module build. Are you saying there's another
way that modules in a multi-module build can share artifacts? Could you
point me to where this is documented?
> of course the real
> solution is to fix your build so that it works with "clean verify" on a
> 'virgin' set of version numbers.
>
To be clear: in a multi-module build, with a root pom, with brand new
version numbers, you're saying the whole build should run just fine when
invoked from the root with "clean verify"?
Just another user trying to do things right,
Best,
Laird
>
--
http://about.me/lairdnelson
Re: Seeking feedback on “Recursive Maven considered harmful”
Posted by Stephen Connolly <st...@gmail.com>.
Current work on Maven compiler plugin version 3.0 is adding support for an
incremental mode.
It is a tricky balance to cut, as if incremental mode is too aggressive,
you compile more than is strictly necessary... and if too lax, you compile
less than is required and people end up having to run "clean" first again.
Another issue is that people abuse the "install" hack far more often than
they should...
In fact it seems most people's default mode with Maven is to go "mvn clean
install".
There are a lot of issues with that, not least being that you are abusing
the local repository cache.
Where this really kicks in is when people have made their build to *rely*
on artifacts being installed in the cache... cue ensuing hilarity when
using the release plugin... "solved" (if even detected) by people changing
to <preparationGoals>clean install</preparationGoals>... of course the real
solution is to fix your build so that it works with "clean verify" on a
'virgin' set of version numbers.
One of your points about partial phases actually misses one of the key
things about Maven that a lot of people have missed... namely that you
don't have to go all the way.
The extension of my "install" hack complaint is the "package" hack
complaint.
If your build requires you to go as far as the "package" phase, then likely
you are missing out on doing it right... might not be your fault, may the
the fault of some plugins you are using... but none the less there are
things you are missing out on.
My hope is that, for example, some of the tricks I am applying with the
jszip.org's maven plugin will enlighten people into the benefits of getting
your build 'right'... for example
$ mvn compile jszip:run
In a multi-module build... now your java classpath is not for the
**/target/*.jar files but for the **/target/classes directories... so you
can have your IDE compile the changed classes, and the classpath scanning
of jszip:run will pick up the changes and reload the servlet container...
With the background compile feature in for example IDEA 12 (I ack that
eclipse's webby plugin does similar to jszip:run... some of my tricks were
picked up from Benjamin) you basically can just write your java and
javascript code and switch back to the browser, hit reload and see the
changes...
If you do "mvn package jszip:run" however the `target/classes` directory
will have been replaced as the primary artifact by the
`target/${finalName}.jar` and therefore we are limited to watching for the
.jar file to be modified... no more live development. Yes I could hack and
infer the target/classes directory... but that hack would not be the Maven
way.
So the phase you invoke depends on your needs.
You are doing Maven wrong (IMHO) if your entire build cannot work under the
following incremental behaviours:
1. Change all the modules to a 'virgin' version number (one that has never
been touched before... don't want the local repo or any remote repos
"helping" us out
2. "mvn clean" if you cannot clean a clean fresh checkout on virgin version
numbers #fail
3. "mvn clean generate-sources" if you cannot create all the generated
source code, your IDE experience will be less than stellar #fail
4. "mvn clean compile" if you cannot compile all your production code
without bundling up jars and stuffing them in the local repo cache, shame
on you #fail
5. "mvn clean test-compile" if you cannot compile all your test code #fail
6. "mvn clean package -DskipTests" if those packaged artifacts need to go
into the local repo cache for subsequent modules to find them, you have a
subtly broken build #fail
7. "mvn clean verify -DskipTests" if you cannot use the release plugin's
default test of a releaseable artifact on virgin version numbers: #fail
Note that all these tests are designed to be quick (i.e. not looking at
whether you have working test cases and ignoring the results of tests) and
to be run in succession on the same virgin version number(s)
Not that you will use all of the above commands on a regular basis... but
if you cannot pass all 6 tests, you are not enabling the full power of
Maven and I personally will not listen to you complaining about how "Maven
sucks" because the truth is *your* build sucks and therefore we cannot make
any inference as to the sucky-ness of Maven (which may indeed suck... but I
have yet to see a build that passes my tests and shows Maven to suck)
-Stephen
On 17 December 2012 10:50, Thomas Broyer <t....@gmail.com> wrote:
> Hi,
>
> On his blog, Lex Spoon makes some interesting points against most
> build systems, using Maven as an example:
> http://blog.lexspoon.org/2012/12/recursive-maven-considered-harmful.html
> In related news, I've heard people complain about “insanely long build
> times via building unnecessary things” and tracking down the cause to
> being Maven, to the extent that they're thinking about making their
> own build system (many are Xooglers and they probably would like.
>
> I've made some proposals for improvements in the comments of Lex's
> post, and I'm eager to know what other people think, particularly
> Maven core developers/contributors. Has this been ever discussed? If
> yes, is this planned or has it been dismissed? In the latter case, for
> which reasons?
>
> Thanks in advance,
>
> --
> Thomas Broyer
> /tɔ.ma.bʁwa.je/ <http://xn--nna.ma.xn--bwa-xxb.je/>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
Re: Seeking feedback on “Recursive Maven considered harmful”
Posted by Anders Hammar <an...@hammar.net>.
Mark Struberg has started work to improve the incremental behavior of e.g.
the compiler plugin. Version 3.0 of maven-compiler-plugin includes an early
version of that new library. There was a thread about this, started by
Mark, on the dev list some ago if you want to read about it.
Mark has also written a wiki page [1] on this work.
/Anders
[1] https://cwiki.apache.org/MAVEN/incremental-builds.html
On Mon, Dec 17, 2012 at 11:50 AM, Thomas Broyer <t....@gmail.com> wrote:
> Hi,
>
> On his blog, Lex Spoon makes some interesting points against most
> build systems, using Maven as an example:
> http://blog.lexspoon.org/2012/12/recursive-maven-considered-harmful.html
> In related news, I've heard people complain about “insanely long build
> times via building unnecessary things” and tracking down the cause to
> being Maven, to the extent that they're thinking about making their
> own build system (many are Xooglers and they probably would like.
>
> I've made some proposals for improvements in the comments of Lex's
> post, and I'm eager to know what other people think, particularly
> Maven core developers/contributors. Has this been ever discussed? If
> yes, is this planned or has it been dismissed? In the latter case, for
> which reasons?
>
> Thanks in advance,
>
> --
> Thomas Broyer
> /tɔ.ma.bʁwa.je/ <http://xn--nna.ma.xn--bwa-xxb.je/>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>