How to Override Maven -T behavior

[Rohit Iyer <ca...@gmail.com>]

Hi,

We are looking for ways to optimize the build time for our project which
has about 36 modules. The Dependency hierarchy takes us to a tree of 10
levels of nodes and some of the modules need mock server testing which runs
on the same port and parallel tests fail when two modules need the mock
server at the same time.

We are exploring alternative to:

1. version project to a temporary version
2. Trigger fast build using -T (skip test=true) which also installs the
artifact to local repo.
3. Then run -T using our own project module order which no longer requires
dependencies because they are available and we can guarantee N/C times
improvement where C is number of available cores.  This helps us avoid the
mock server dependency if we ensure all of them run in the same thread
/core sequentially.

Can you suggest to us which plugin we need to modify and use?

Thanks
Rohit

Re: How to Override Maven -T behavior

[Karl Heinz Marbaise <kh...@gmx.de>]

Hi,

On 5/22/16 8:39 AM, Rohit Iyer wrote:
> Hi,
>
> We are looking for ways to optimize the build time for our project which
> has about 36 modules. The Dependency hierarchy takes us to a tree of 10
> levels of nodes and some of the modules need mock server testing which runs
> on the same port and parallel tests fail when two modules need the mock
> server at the same time.

First 36 Modules is a small build (I'm on projects with 500 modules 
which took about 15 mins with 10000 unit tests but only has 5 levels and 
around 750,000 lines of code)...furthermore what do you mean by 
optimize? In which way? Time ?

How long does it take? Build Time / Test Time etc. How many lines of 
code do you have ? The number of tests?

Apart from that the dependency hierarchy based on the interdependencies 
between the modules you have? Or do you mean the dependencies the module 
have ?

Do your mock server tests run in test life cycle or shouldn't they run 
in integration-test phase (pre-integration-test, integration-test, 
post-integration-test)? By default integration tests can't be parallized 
..unit tests can cause they are independent...

If you really have 10 levels for 36 modules sounds like your 
architecture could be improved ?...

>
> We are exploring alternative to:
>
> 1. version project to a temporary version

What do you mean by that?


> 2. Trigger fast build using -T (skip test=true) which also installs the
> artifact to local repo.


> 3. Then run -T using our own project module order which no longer requires
> dependencies because they are available and we can guarantee N/C times
> improvement where C is number of available cores.

Why are you trying to handle the order manually does not make sense 
maven takes care of that...

And you should be aware that there is a tradeoff between the number of 
cores and the time a build take ...for example:

mvn -T 2 ..

speed time up

mvn -T 10 ...

get slower...

so the optimal way is somewhere between those values...


 >
   This helps us avoid the
> mock server dependency if we ensure all of them run in the same thread
> /core sequentially.

Mock server sounds like integration tests which either shouldn't be run 
at that time or the tests or the mock server must handle that to use 
different ports...but this is an indicator that those tests are 
integration tests..


So i would suggest to separate unit/integration test by using the 
correct plugins/phase (maven-surefire-plugin for unit tests and 
maven-failsafe-plugin for running integration tests following the naming 
conventions or may be it's needed to have separated modules which 
contain the integration tests (sometimes usefull)...)..

May be it's also worth to think about running only compiling etc. and 
unit tests in one and separating the integration tests into a different 
run...

A CI Server is very helpfull in such situations...also using pipelines...



>
> Can you suggest to us which plugin we need to modify and use?



>
> Thanks
> Rohit
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org