You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Hanmay Udgiri <ha...@gmail.com> on 2012/09/17 09:17:29 UTC
Improving project build time
Hi All,
We are Maven for building and deploying our projects.
The total time taken for the project is around 32mins.
We want to improve this time.
Please suggest different ways of reducing the build time.
The projects contains total 22 module.
when we run the mvn clean and install without tests it takes around 1 min
22 seconds.
But when we run with Tests it takes around 32 mins.
--
Thanks and Regards
Hanmayya Udgiri
Re: Improving project build time
Posted by John Kramer <jk...@mojiva.com>.
I would suggest that you make sure that your tests are idempotent
(http://geekswithblogs.net/dthakur/archive/2004/11/19/15282.aspx) and
independent and deterministic. This will not only allow you to run tests
in parallel, but it will also help maintainability a lot.
It is also a red flag that you need to cal thread.sleep in your tests.
Are you testing multithreaded processes? If to, it is probably better to
use thread.wait and thread.notify.
John Kramer
email: jkramer@mojiva.com
mobile: 314.435.2370
skype: kramer.mojiva
twitter: @KramerKnowsTech <https://twitter.com/KramerKnowsTech>
0xCAFEBABE00000032
On 9/17/12 10:17 AM, "Hanmay Udgiri" <ha...@gmail.com> wrote:
>thx guys for your comments..
>yes we have integration tests included in the build.
>I have tried running them parallel but the tests are not atomic so are not
>working as expected.
>we have some DB call where we are using thread sleep,is there anything by
>which this can be improved
>Also we have any best practices for maven builds.
>
>Thanks and Regards
>Hanmayya
>
>On Mon, Sep 17, 2012 at 6:48 PM, John Kramer <jk...@mojiva.com> wrote:
>
>> I understand your point, and agree that I might be nice to have a bit
>>more
>> of a standard place for integration tests. However, one point that
>>should
>> be mentioned is that there are frequently multiple levels of integration
>> tests (in container, out of container, ui, etc). Thus it might not be
>>so
>> simple to just have one spot for integration tests.
>>
>> John Kramer
>> email: jkramer@mojiva.com
>> mobile: 314.435.2370
>> skype: kramer.mojiva
>> twitter: @KramerKnowsTech <https://twitter.com/KramerKnowsTech>
>> 0xCAFEBABE00000032
>>
>>
>>
>>
>>
>>
>>
>> On 9/17/12 9:05 AM, "Clebert Suconic" <cl...@gmail.com> wrote:
>>
>> >That's one thing I always thought that maven should have an explicit
>> >spot for integration tests. We also had to create a separate profile.
>> >Having them into a more standard place would make it cleaner IMO.
>> >
>> >That's a feature request I know. But does it make sense?
>> >
>> >Sent from my iPhone
>> >
>> >On Sep 17, 2012, at 5:39 AM, Barrie Treloar <ba...@gmail.com> wrote:
>> >
>> >> On Mon, Sep 17, 2012 at 5:26 PM, Kristian Rosenvold
>> >> <kr...@gmail.com> wrote:
>> >>> You should probably be looking into parallelizing your tests :) .
>> >>
>> >> Or looking at what your tests are doing.
>> >> 32 minutes sounds like you are doing integration type testing and not
>> >> unit tests.
>> >>
>> >> Pull the integration tests into their own module, which you include
>> >> via a profile (e.g. "its"), and get your continuous integration
>>server
>> >> to run them for you.
>> >> The developers can then forget about them until they get emailed
>>about
>> >> broken builds.
>> >>
>> >> If they really want to run them, they can just include them via -Pits
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> >> For additional commands, e-mail: users-help@maven.apache.org
>> >>
>> >
>> >---------------------------------------------------------------------
>> >To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> >For additional commands, e-mail: users-help@maven.apache.org
>> >
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>>
>>
>
>
>--
>Thanks and Regards
>Hanmayya Udgiri
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Improving project build time
Posted by Hanmay Udgiri <ha...@gmail.com>.
thx guys for your comments..
yes we have integration tests included in the build.
I have tried running them parallel but the tests are not atomic so are not
working as expected.
we have some DB call where we are using thread sleep,is there anything by
which this can be improved
Also we have any best practices for maven builds.
Thanks and Regards
Hanmayya
On Mon, Sep 17, 2012 at 6:48 PM, John Kramer <jk...@mojiva.com> wrote:
> I understand your point, and agree that I might be nice to have a bit more
> of a standard place for integration tests. However, one point that should
> be mentioned is that there are frequently multiple levels of integration
> tests (in container, out of container, ui, etc). Thus it might not be so
> simple to just have one spot for integration tests.
>
> John Kramer
> email: jkramer@mojiva.com
> mobile: 314.435.2370
> skype: kramer.mojiva
> twitter: @KramerKnowsTech <https://twitter.com/KramerKnowsTech>
> 0xCAFEBABE00000032
>
>
>
>
>
>
>
> On 9/17/12 9:05 AM, "Clebert Suconic" <cl...@gmail.com> wrote:
>
> >That's one thing I always thought that maven should have an explicit
> >spot for integration tests. We also had to create a separate profile.
> >Having them into a more standard place would make it cleaner IMO.
> >
> >That's a feature request I know. But does it make sense?
> >
> >Sent from my iPhone
> >
> >On Sep 17, 2012, at 5:39 AM, Barrie Treloar <ba...@gmail.com> wrote:
> >
> >> On Mon, Sep 17, 2012 at 5:26 PM, Kristian Rosenvold
> >> <kr...@gmail.com> wrote:
> >>> You should probably be looking into parallelizing your tests :) .
> >>
> >> Or looking at what your tests are doing.
> >> 32 minutes sounds like you are doing integration type testing and not
> >> unit tests.
> >>
> >> Pull the integration tests into their own module, which you include
> >> via a profile (e.g. "its"), and get your continuous integration server
> >> to run them for you.
> >> The developers can then forget about them until they get emailed about
> >> broken builds.
> >>
> >> If they really want to run them, they can just include them via -Pits
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> >> For additional commands, e-mail: users-help@maven.apache.org
> >>
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> >For additional commands, e-mail: users-help@maven.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
--
Thanks and Regards
Hanmayya Udgiri
Re: Improving project build time
Posted by Clebert Suconic <cl...@gmail.com>.
Option one:
You could have one separate project for each integration layer?
I was just looking into something like, I just arrived in a project.. want
to run the integration tests... then I could just do:
cd /proper-integration-directory
mvn integration-tests
mvn tests:integration-tests
option 2:
mvn tests:module-name (where module name would be the layer you mentioned)
Anyway: I understand there are other priorities so this will be a minor
thing. I was just proposing a standard way (from my user's POV).
RE: Improving project build time
Posted by Danny Thomas <Da...@blackboard.com>.
Use the Failsafe plugin, though a separate module is good practice too:
http://maven.apache.org/plugins/maven-failsafe-plugin/
Danny
-----Original Message-----
From: Clebert Suconic [mailto:clebert.suconic@gmail.com]
Sent: 17 September 2012 14:05
To: Maven Users List
Subject: Re: Improving project build time
That's one thing I always thought that maven should have an explicit spot for integration tests. We also had to create a separate profile.
Having them into a more standard place would make it cleaner IMO.
That's a feature request I know. But does it make sense?
Sent from my iPhone
On Sep 17, 2012, at 5:39 AM, Barrie Treloar <ba...@gmail.com> wrote:
> On Mon, Sep 17, 2012 at 5:26 PM, Kristian Rosenvold
> <kr...@gmail.com> wrote:
>> You should probably be looking into parallelizing your tests :) .
>
> Or looking at what your tests are doing.
> 32 minutes sounds like you are doing integration type testing and not
> unit tests.
>
> Pull the integration tests into their own module, which you include
> via a profile (e.g. "its"), and get your continuous integration server
> to run them for you.
> The developers can then forget about them until they get emailed about
> broken builds.
>
> If they really want to run them, they can just include them via -Pits
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
This email and any attachments may contain confidential and proprietary information of Blackboard that is for the sole use of the intended recipient. If you are not the intended recipient, disclosure, copying, re-distribution or other use of any of this information is strictly prohibited. Please immediately notify the sender and delete this transmission if you received this email in error.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Improving project build time
Posted by John Kramer <jk...@mojiva.com>.
I understand your point, and agree that I might be nice to have a bit more
of a standard place for integration tests. However, one point that should
be mentioned is that there are frequently multiple levels of integration
tests (in container, out of container, ui, etc). Thus it might not be so
simple to just have one spot for integration tests.
John Kramer
email: jkramer@mojiva.com
mobile: 314.435.2370
skype: kramer.mojiva
twitter: @KramerKnowsTech <https://twitter.com/KramerKnowsTech>
0xCAFEBABE00000032
On 9/17/12 9:05 AM, "Clebert Suconic" <cl...@gmail.com> wrote:
>That's one thing I always thought that maven should have an explicit
>spot for integration tests. We also had to create a separate profile.
>Having them into a more standard place would make it cleaner IMO.
>
>That's a feature request I know. But does it make sense?
>
>Sent from my iPhone
>
>On Sep 17, 2012, at 5:39 AM, Barrie Treloar <ba...@gmail.com> wrote:
>
>> On Mon, Sep 17, 2012 at 5:26 PM, Kristian Rosenvold
>> <kr...@gmail.com> wrote:
>>> You should probably be looking into parallelizing your tests :) .
>>
>> Or looking at what your tests are doing.
>> 32 minutes sounds like you are doing integration type testing and not
>> unit tests.
>>
>> Pull the integration tests into their own module, which you include
>> via a profile (e.g. "its"), and get your continuous integration server
>> to run them for you.
>> The developers can then forget about them until they get emailed about
>> broken builds.
>>
>> If they really want to run them, they can just include them via -Pits
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>For additional commands, e-mail: users-help@maven.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Improving project build time
Posted by Clebert Suconic <cl...@gmail.com>.
That's one thing I always thought that maven should have an explicit
spot for integration tests. We also had to create a separate profile.
Having them into a more standard place would make it cleaner IMO.
That's a feature request I know. But does it make sense?
Sent from my iPhone
On Sep 17, 2012, at 5:39 AM, Barrie Treloar <ba...@gmail.com> wrote:
> On Mon, Sep 17, 2012 at 5:26 PM, Kristian Rosenvold
> <kr...@gmail.com> wrote:
>> You should probably be looking into parallelizing your tests :) .
>
> Or looking at what your tests are doing.
> 32 minutes sounds like you are doing integration type testing and not
> unit tests.
>
> Pull the integration tests into their own module, which you include
> via a profile (e.g. "its"), and get your continuous integration server
> to run them for you.
> The developers can then forget about them until they get emailed about
> broken builds.
>
> If they really want to run them, they can just include them via -Pits
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Improving project build time
Posted by Barrie Treloar <ba...@gmail.com>.
On Mon, Sep 17, 2012 at 5:26 PM, Kristian Rosenvold
<kr...@gmail.com> wrote:
> You should probably be looking into parallelizing your tests :) .
Or looking at what your tests are doing.
32 minutes sounds like you are doing integration type testing and not
unit tests.
Pull the integration tests into their own module, which you include
via a profile (e.g. "its"), and get your continuous integration server
to run them for you.
The developers can then forget about them until they get emailed about
broken builds.
If they really want to run them, they can just include them via -Pits
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Improving project build time
Posted by Kristian Rosenvold <kr...@gmail.com>.
You should probably be looking into parallelizing your tests :) .
Either through using the parallel attribute of surefire or or
forkMode=perThread.
( see http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html)
forkMode=perThread is probably the easiest to get going with, but the
payoff is larger
with the parallel flag. You only need to watch out for singleton stutt
(tcp ports, files)
when trying to run parallel.
Kristian
2012/9/17 Hanmay Udgiri <ha...@gmail.com>:
> Hi All,
>
> We are Maven for building and deploying our projects.
>
> The total time taken for the project is around 32mins.
>
> We want to improve this time.
> Please suggest different ways of reducing the build time.
>
> The projects contains total 22 module.
> when we run the mvn clean and install without tests it takes around 1 min
> 22 seconds.
> But when we run with Tests it takes around 32 mins.
>
>
> --
> Thanks and Regards
> Hanmayya Udgiri
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org