Parallelization and skip compilation

["Subramanian, N.Venkata" <ns...@informatica.com>]

Hi 

          Good morning!

 

I have couple of question which is quite critical in our current project
scenario

 

0) Our product is supporting around 14 platforms and our prodct code is
divided into many components and subcomponents. We have big issue that
for most of the platforms the build is taking too much time (10-14 hrs
for the whole buid life cycle ) and we need to reduce the build 

 

Why this question is posted on maven forum than a s/w build forum.
Ofcourse our builds are based on maven. 

But that's not the only reason for me to post this. I would like to know


-         what are the possible ways to parallelize the maven component
(and sub components)

-         if we can parallelize, how to identify depedencies between the
components so that we can parallelize the *independent* components

to avoid the conflicts across components while building. [we have mvn
dependency:tree - but that is giving dependency information at the very
granular level instead of at the top level]

 

 

- how to skip compilation in maven - [ this question is being asked many
times. ] I am not sure why this feature is not available. We have
scenario that    We first compile and test, if it succeeds then we
deploy on the same workspace. At this instance we don't need to really
compile by which we can save around 50% of the deploy build time.

 

 

Appreciate your help!

Thanks

-Venkat

Re: Parallelization and skip compilation

[Tim O'Brien <to...@discursive.com>]

On Tue, Jul 14, 2009 at 1:40 PM, Subramanian,
N.Venkata<ns...@informatica.com> wrote:
> Hi
>
>          Good morning!
>
>
>
> I have couple of question which is quite critical in our current project
> scenario
>
>
>
> 0) Our product is supporting around 14 platforms and our prodct code is
> divided into many components and subcomponents. We have big issue that
> for most of the platforms the build is taking too much time (10-14 hrs
> for the whole buid life cycle ) and we need to reduce the build
>
>
>
> Why this question is posted on maven forum than a s/w build forum.
> Ofcourse our builds are based on maven.
>
> But that's not the only reason for me to post this. I would like to know
>
>
> -         what are the possible ways to parallelize the maven component
> (and sub components)

You could always build a parent project with the -N flag, this would
have the effect of installing a parent POM.    Then you could go into
submodules and kick off a separate build in each module (as long as
there are no cross-dependencies between submodules).   Usually you are
going to have interdependent submodules, so you'll have to figure out
a way to create isolated trees of submodules - I'll leave that
exercise up to you because I don't know anything about the
architecture of your multi-module project.

A better solution to support multiple platforms would be to setup a
number of Hudson builds running across a grid of machines.   This is
what happens on grid.sonatype.org - take a look at the core
integration tests for Maven 3:
https://grid.sonatype.org/ci/view/Maven%203.0.x/job/maven-core-integration-testing/
   - these tests are run on different nodes of a grid of build
machines managed by Hudson.

>
> -         if we can parallelize, how to identify depedencies between the
> components so that we can parallelize the *independent* components
>
> to avoid the conflicts across components while building. [we have mvn
> dependency:tree - but that is giving dependency information at the very
> granular level instead of at the top level]
>

Ok, you are right, dependency:tree is always going to show you the
resolved dependencies for a single project.    What you are looking
for is something that would step through project dependencies in your
tree.   You will likely want to take a look at the Maven Reactor
plugin as a diagnostic tool that would point you in the right
direction.

http://maven.apache.org/plugins/maven-reactor-plugin/make-dependents-mojo.html

Run reactor:make and reactor:make-dependents and pass in
-DprintOnly=true  - this will shed some light on module dependencies
within your multimodule project.

>
>
>
>
> - how to skip compilation in maven - [ this question is being asked many
> times. ] I am not sure why this feature is not available. We have
> scenario that    We first compile and test, if it succeeds then we
> deploy on the same workspace. At this instance we don't need to really
> compile by which we can save around 50% of the deploy build time.
>

Ok, so you want to just run deploy?     When you run "mvn deploy" it
is running every lifecycle stage up to deploy (including compile).
My advice, if you really want to omit compilation would be to
explictly run the goals you need.  It isn't pretty, but you could just
run "mvn deploy:deploy"

Take a look at this page about the lifecycle: http://tr.im/shS4 - look
at the default bindings to the default maven lifecycle and just pick
and choose which goals you want to run.  If you wanted to run the
final two stages, but you want to skip the lifecycle, you could run
"mvn install:install deploy:deploy".  It is unorthodox, but it might
be able to get you what you need.

>
>
>
>
> Appreciate your help!
>
> Thanks
>
> -Venkat
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org