Maven bootstrap/surefire:test performance

[Mirko Friedenhagen <mf...@gmail.com>]

Hello everybody,

I use Netbeans 7.4 as IDE and mostly like how it uses Maven to get stuff
done. While it is nice that I do not encounter problems as I did with
Eclipse when dealing with dependency scoping I am slowed down when running
single test files or methods. Netbeans invokes "test-compile surefire:test
-Dtest=...".

Now even from the CLI it takes Maven some time to reach surefire:test. Even
when I do not invoke test-compile 3 seconds are spent before surefire:test
starts it's work and another 3 seconds afterwards.

The tests of a single testcase themselves only take 0.3 seconds. With
Eclipse or Intellij the execution happens in less than 1 second while with
Maven or Netbeans this will take up to 10 seconds even in offline mode.

Any hints for speeding up things would be appreciated (except of changing
the IDE ;-)).

Regards Mirko
-- 
Sent from my mobile

Re: Maven bootstrap/surefire:test performance

[Milos Kleint <mk...@gmail.com>]

that's the maven/plexus/guice boostrap IMHO.

Milos

On Sat, Sep 14, 2013 at 9:06 PM, Mirko Friedenhagen <mfriedenhagen@gmail.com
> wrote:

> Mark,
>
> I like the decision to use Maven whenever possible as well, nonetheless I
> could imagine being even more happy should the execution times decrease. I
> use a very small maven projects for tests[0] with Maven 3.1.1, turned on
> timestamps and it takes Maven about 2 seconds when running "mvn -o
> surefire:test". Even with option "-X" I do not see what is happening in the
> first 1300ms, the first output line "1775 [INFO] Scanning for projects..."
> starts after 1800ms.
>
> Regards Mirko
>
> [0] https://github.com/mfriedenhagen/pastebin/tree/surefire-speed-test
> --
> Sent from my mobile
> On Sep 13, 2013 9:49 PM, "Mark Eggers" <it...@yahoo.com> wrote:
>
> > On 9/13/2013 12:38 PM, Milos Kleint wrote:
> >
> >> with Compile on Save enabled, the test-compile phase should be skipped
> and
> >> only surefire:test should be executed.
> >>
> >> However even with that, there's overhead of jvm startup + maven startup
> >> before the mojo gets executed. Obviously the overhead is biggest when
> you
> >> run just a single test. Unfortunately not much that can be done here.
> >> We've
> >> used to execute "Compile on Save" stuff with internal nb execution (via
> >> ant
> >> in IDE-jvm) but the devil is in the detail there, it's not 100% exactly
> >> the
> >> same execution as surefire. The current design decision is to be 100%
> >> equal
> >> to what cmd line executes. The downside is speed.
> >>
> >> Milos
> >>
> >>
> >> On Fri, Sep 13, 2013 at 9:25 PM, Mirko Friedenhagen <
> >> mfriedenhagen@gmail.com
> >>
> >>> wrote:
> >>>
> >>
> >>  Hello everybody,
> >>>
> >>> I use Netbeans 7.4 as IDE and mostly like how it uses Maven to get
> stuff
> >>> done. While it is nice that I do not encounter problems as I did with
> >>> Eclipse when dealing with dependency scoping I am slowed down when
> >>> running
> >>> single test files or methods. Netbeans invokes "test-compile
> >>> surefire:test
> >>> -Dtest=...".
> >>>
> >>> Now even from the CLI it takes Maven some time to reach surefire:test.
> >>> Even
> >>> when I do not invoke test-compile 3 seconds are spent before
> >>> surefire:test
> >>> starts it's work and another 3 seconds afterwards.
> >>>
> >>> The tests of a single testcase themselves only take 0.3 seconds. With
> >>> Eclipse or Intellij the execution happens in less than 1 second while
> >>> with
> >>> Maven or Netbeans this will take up to 10 seconds even in offline mode.
> >>>
> >>> Any hints for speeding up things would be appreciated (except of
> changing
> >>> the IDE ;-)).
> >>>
> >>> Regards Mirko
> >>> --
> >>> Sent from my mobile
> >>>
> >>>
> >>
> > Yep, with Compile on Save and an external Maven (using 3.1) and a
> > reasonably recent NetBeans 7.4 build I get about 3 seconds for executing
> a
> > focused test method.
> >
> > In short, it's not the 10 seconds you're seeing, but it's not the 0.3
> > seconds you're expecting.
> >
> > I can certainly live with (and in fact applaud) the design decision to be
> > 100% equal to what the command line executes.
> >
> > . . . . just my two cents
> > /mde/
> >
> > ------------------------------**------------------------------**---------
> > To unsubscribe, e-mail: users-unsubscribe@maven.**apache.org<
> users-unsubscribe@maven.apache.org>
> > For additional commands, e-mail: users-help@maven.apache.org
> >
> >
>

Re: Maven bootstrap/surefire:test performance

[Mirko Friedenhagen <mf...@gmail.com>]

Mark,

I like the decision to use Maven whenever possible as well, nonetheless I
could imagine being even more happy should the execution times decrease. I
use a very small maven projects for tests[0] with Maven 3.1.1, turned on
timestamps and it takes Maven about 2 seconds when running "mvn -o
surefire:test". Even with option "-X" I do not see what is happening in the
first 1300ms, the first output line "1775 [INFO] Scanning for projects..."
starts after 1800ms.

Regards Mirko

[0] https://github.com/mfriedenhagen/pastebin/tree/surefire-speed-test
-- 
Sent from my mobile
On Sep 13, 2013 9:49 PM, "Mark Eggers" <it...@yahoo.com> wrote:

> On 9/13/2013 12:38 PM, Milos Kleint wrote:
>
>> with Compile on Save enabled, the test-compile phase should be skipped and
>> only surefire:test should be executed.
>>
>> However even with that, there's overhead of jvm startup + maven startup
>> before the mojo gets executed. Obviously the overhead is biggest when you
>> run just a single test. Unfortunately not much that can be done here.
>> We've
>> used to execute "Compile on Save" stuff with internal nb execution (via
>> ant
>> in IDE-jvm) but the devil is in the detail there, it's not 100% exactly
>> the
>> same execution as surefire. The current design decision is to be 100%
>> equal
>> to what cmd line executes. The downside is speed.
>>
>> Milos
>>
>>
>> On Fri, Sep 13, 2013 at 9:25 PM, Mirko Friedenhagen <
>> mfriedenhagen@gmail.com
>>
>>> wrote:
>>>
>>
>>  Hello everybody,
>>>
>>> I use Netbeans 7.4 as IDE and mostly like how it uses Maven to get stuff
>>> done. While it is nice that I do not encounter problems as I did with
>>> Eclipse when dealing with dependency scoping I am slowed down when
>>> running
>>> single test files or methods. Netbeans invokes "test-compile
>>> surefire:test
>>> -Dtest=...".
>>>
>>> Now even from the CLI it takes Maven some time to reach surefire:test.
>>> Even
>>> when I do not invoke test-compile 3 seconds are spent before
>>> surefire:test
>>> starts it's work and another 3 seconds afterwards.
>>>
>>> The tests of a single testcase themselves only take 0.3 seconds. With
>>> Eclipse or Intellij the execution happens in less than 1 second while
>>> with
>>> Maven or Netbeans this will take up to 10 seconds even in offline mode.
>>>
>>> Any hints for speeding up things would be appreciated (except of changing
>>> the IDE ;-)).
>>>
>>> Regards Mirko
>>> --
>>> Sent from my mobile
>>>
>>>
>>
> Yep, with Compile on Save and an external Maven (using 3.1) and a
> reasonably recent NetBeans 7.4 build I get about 3 seconds for executing a
> focused test method.
>
> In short, it's not the 10 seconds you're seeing, but it's not the 0.3
> seconds you're expecting.
>
> I can certainly live with (and in fact applaud) the design decision to be
> 100% equal to what the command line executes.
>
> . . . . just my two cents
> /mde/
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: users-unsubscribe@maven.**apache.org<us...@maven.apache.org>
> For additional commands, e-mail: users-help@maven.apache.org
>
>

Re: Maven bootstrap/surefire:test performance

[Milos Kleint <mk...@gmail.com>]

On Fri, Sep 13, 2013 at 9:47 PM, Mark Eggers <it...@yahoo.com> wrote:

> On 9/13/2013 12:38 PM, Milos Kleint wrote:
>
>> with Compile on Save enabled, the test-compile phase should be skipped and
>> only surefire:test should be executed.
>>
>> However even with that, there's overhead of jvm startup + maven startup
>> before the mojo gets executed. Obviously the overhead is biggest when you
>> run just a single test. Unfortunately not much that can be done here.
>> We've
>> used to execute "Compile on Save" stuff with internal nb execution (via
>> ant
>> in IDE-jvm) but the devil is in the detail there, it's not 100% exactly
>> the
>> same execution as surefire. The current design decision is to be 100%
>> equal
>> to what cmd line executes. The downside is speed.
>>
>> Milos
>>
>>
>> On Fri, Sep 13, 2013 at 9:25 PM, Mirko Friedenhagen <
>> mfriedenhagen@gmail.com
>>
>>> wrote:
>>>
>>
>>  Hello everybody,
>>>
>>> I use Netbeans 7.4 as IDE and mostly like how it uses Maven to get stuff
>>> done. While it is nice that I do not encounter problems as I did with
>>> Eclipse when dealing with dependency scoping I am slowed down when
>>> running
>>> single test files or methods. Netbeans invokes "test-compile
>>> surefire:test
>>> -Dtest=...".
>>>
>>> Now even from the CLI it takes Maven some time to reach surefire:test.
>>> Even
>>> when I do not invoke test-compile 3 seconds are spent before
>>> surefire:test
>>> starts it's work and another 3 seconds afterwards.
>>>
>>> The tests of a single testcase themselves only take 0.3 seconds. With
>>> Eclipse or Intellij the execution happens in less than 1 second while
>>> with
>>> Maven or Netbeans this will take up to 10 seconds even in offline mode.
>>>
>>> Any hints for speeding up things would be appreciated (except of changing
>>> the IDE ;-)).
>>>
>>> Regards Mirko
>>> --
>>> Sent from my mobile
>>>
>>>
>>
> Yep, with Compile on Save and an external Maven (using 3.1) and a
> reasonably recent NetBeans 7.4 build I get about 3 seconds for executing a
> focused test method.
>
> In short, it's not the 10 seconds you're seeing, but it's not the 0.3
> seconds you're expecting.
>
> I can certainly live with (and in fact applaud) the design decision to be
> 100% equal to what the command line executes.
>
>
actually the biggest problem I get to hear about is not the speed (which I
admit is slower than expected), but the fact that run focused method often
doesn't work, that's due to various combinations of maven-surefire-plugin
and junit/testng version. Only some combinations support the executions of
single test methods.

Milos


> . . . . just my two cents
> /mde/
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: users-unsubscribe@maven.**apache.org<us...@maven.apache.org>
> For additional commands, e-mail: users-help@maven.apache.org
>
>

Re: Maven bootstrap/surefire:test performance

[Mark Eggers <it...@yahoo.com>]

On 9/13/2013 12:38 PM, Milos Kleint wrote:
> with Compile on Save enabled, the test-compile phase should be skipped and
> only surefire:test should be executed.
>
> However even with that, there's overhead of jvm startup + maven startup
> before the mojo gets executed. Obviously the overhead is biggest when you
> run just a single test. Unfortunately not much that can be done here. We've
> used to execute "Compile on Save" stuff with internal nb execution (via ant
> in IDE-jvm) but the devil is in the detail there, it's not 100% exactly the
> same execution as surefire. The current design decision is to be 100% equal
> to what cmd line executes. The downside is speed.
>
> Milos
>
>
> On Fri, Sep 13, 2013 at 9:25 PM, Mirko Friedenhagen <mfriedenhagen@gmail.com
>> wrote:
>
>> Hello everybody,
>>
>> I use Netbeans 7.4 as IDE and mostly like how it uses Maven to get stuff
>> done. While it is nice that I do not encounter problems as I did with
>> Eclipse when dealing with dependency scoping I am slowed down when running
>> single test files or methods. Netbeans invokes "test-compile surefire:test
>> -Dtest=...".
>>
>> Now even from the CLI it takes Maven some time to reach surefire:test. Even
>> when I do not invoke test-compile 3 seconds are spent before surefire:test
>> starts it's work and another 3 seconds afterwards.
>>
>> The tests of a single testcase themselves only take 0.3 seconds. With
>> Eclipse or Intellij the execution happens in less than 1 second while with
>> Maven or Netbeans this will take up to 10 seconds even in offline mode.
>>
>> Any hints for speeding up things would be appreciated (except of changing
>> the IDE ;-)).
>>
>> Regards Mirko
>> --
>> Sent from my mobile
>>
>

Yep, with Compile on Save and an external Maven (using 3.1) and a 
reasonably recent NetBeans 7.4 build I get about 3 seconds for executing 
a focused test method.

In short, it's not the 10 seconds you're seeing, but it's not the 0.3 
seconds you're expecting.

I can certainly live with (and in fact applaud) the design decision to 
be 100% equal to what the command line executes.

. . . . just my two cents
/mde/

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org

Re: Maven bootstrap/surefire:test performance

[Milos Kleint <mk...@gmail.com>]

with Compile on Save enabled, the test-compile phase should be skipped and
only surefire:test should be executed.

However even with that, there's overhead of jvm startup + maven startup
before the mojo gets executed. Obviously the overhead is biggest when you
run just a single test. Unfortunately not much that can be done here. We've
used to execute "Compile on Save" stuff with internal nb execution (via ant
in IDE-jvm) but the devil is in the detail there, it's not 100% exactly the
same execution as surefire. The current design decision is to be 100% equal
to what cmd line executes. The downside is speed.

Milos


On Fri, Sep 13, 2013 at 9:25 PM, Mirko Friedenhagen <mfriedenhagen@gmail.com
> wrote:

> Hello everybody,
>
> I use Netbeans 7.4 as IDE and mostly like how it uses Maven to get stuff
> done. While it is nice that I do not encounter problems as I did with
> Eclipse when dealing with dependency scoping I am slowed down when running
> single test files or methods. Netbeans invokes "test-compile surefire:test
> -Dtest=...".
>
> Now even from the CLI it takes Maven some time to reach surefire:test. Even
> when I do not invoke test-compile 3 seconds are spent before surefire:test
> starts it's work and another 3 seconds afterwards.
>
> The tests of a single testcase themselves only take 0.3 seconds. With
> Eclipse or Intellij the execution happens in less than 1 second while with
> Maven or Netbeans this will take up to 10 seconds even in offline mode.
>
> Any hints for speeding up things would be appreciated (except of changing
> the IDE ;-)).
>
> Regards Mirko
> --
> Sent from my mobile
>