You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Paul Hammant <pa...@hammant.org> on 2017/02/16 10:06:37 UTC
Configuring the surefire plugin - a question
Hi folks,.
I've a fast WebDriver using build that I blogged about: A 16 Second Java
Webapp Build (Including WebDriver Tests)
<http://paulhammant.com/2017/02/05/a-16-second-java-webapp-build-including-webdriver-tests/>
.
Jooby (like SpringBoot and SparkJava) give new options for testing - it can
be instantiated in a JUnit test. Everything can be done in Surefire now,
and the Failsafe plugin isn't needed for these. Don't believe me - watch
the video in the blog entry above, it's not long.
New problem. I want unit tests to run in this order:
1. unit
2. integration (may invoke service calls headlessly)
3. function (will use WebDriver)
I can't work out what magic I have to do with executions to allow that to
happen.
Here is how far I got:
https://github.com/paul-hammant/todobackend-jooby/blob/master/pom.xml#L74
It is all a bit second class, because I'd have to do ...
mvn clean test -Punit-tests
mvn test -Pintegration-tests
mvn test -Pfunctional-tests
... to simulate a pipeline, and I would be happy to just rely on
annotations for classifications.
I really want to do ...
mvn clean test -DexecutionOrder=unit,integration,functional
-DstopBuildAtExecutionBoundariesForTestFailures
... and shave seconds off the build.
How do I configure that tersely and elegantly in the surefire plugin today?
Re: Configuring the surefire plugin - a question
Posted by Paul Hammant <pa...@hammant.org>.
I tried many combinations but couldn't make it shorter with the same behavior :(
- ph ---
Just a small hint about the concept of includes/excludes.
A test is executed if and only if it matches any include (or all if there
are no includes) AND doesn't match any exclude.
The problem I see is that by default you exclude every test (**/*). This
implies that all includes are useless. By overriding the excludes per
execution-block you kind of fixed that.
Robert
.
Re: Configuring the surefire plugin - a question
Posted by Robert Scholte <rf...@apache.org>.
On Fri, 24 Feb 2017 04:32:17 +0100, Paul Hammant <pa...@hammant.org> wrote:
> Thanks João, thanks Robert.
>
> I've taken Robert's snippet and expanded it a little to do what I want:
>
> Diff:
> https://github.com/paul-hammant/todobackend-jooby/commit/9626a3155eddbaea74bbf66a3e899b81227842ee
> (repo: paul-hammant/todobackend-jooby* branch: expectations*)
>
> I found that I had to be explicit about excludes too, and have a
> precursor
> exclude that is outside of the three executions.
>
> I did a bunch of trial and error, but this was the minima.
>
> The test of correctness:
>
> $ mvn install | grep "Time elapsed"
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.237
> sec
> - in todobackend.TodoUnitTest
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.51
> sec
> - in todobackend.TodoIntegrationTest
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 11.608
> sec - in todobackend.TodoWebDriverTest
> (ignore some WebDriver noise to std-err)
> I can't say I really understand the rules about additive includes and
> excludes, but my job is to make speedy builds, not fully understand every
> angle bracket of Maven.
Just a small hint about the concept of includes/excludes.
A test is executed if and only if it matches any include (or all if there
are no includes) AND doesn't match any exclude.
The problem I see is that by default you exclude every test (**/*). This
implies that all includes are useless. By overriding the excludes per
execution-block you kind of fixed that.
Robert
>
> Regards,
>
> - Paul
>
>
> On Fri, Feb 17, 2017 at 2:00 PM, Robert Scholte <rf...@apache.org>
> wrote:
>
>> When you only want to change the pom (not the tests), a set of
>> executionblocks will do the trick:
>>
>> <plugin>
>> <groupId>org.apache.maven.plugins</groupId>
>> <artifactId>maven-surefire-plugin</artifactId>
>> <version>2.18.1</version>
>> <executions>
>> <execution>
>> <id>unit-tests</id>
>> <configuration>
>> <includes>
>> <include>**/*Unit*.java</include>
>> </includes>
>> </configuration>
>> </execution>
>> <execution>
>> <id>functional-tests</id>
>> <configuration>
>> <includes>
>> <include>**/*WebDriver*.java</include>
>> </includes>
>> </configuration>
>> </execution>
>> <execution>
>> <id>integration-tests</id>
>> <configuration>
>> <includes>
>> <include>**/*Integration*.java</include>
>> </includes>
>> </configuration>
>> </execution>
>> <executions>
>> </plugin>
>>
>>
>> On Thu, 16 Feb 2017 11:22:37 +0100, João Cabrita
>> <jo...@gmail.com>
>> wrote:
>>
>> I'd say you could add executions to the surefire plugin with different
>>> categories:
>>> https://maven.apache.org/surefire/maven-surefire-plugin/
>>> examples/junit.html#Using_JUnit_Categories
>>>
>>> Look at this gist (I've omitted some details):
>>> https://gist.github.com/kewne/2b909ab5e8035a4e44e406fa35e3276c
>>>
>>> AFAIK, even if the executions are all bound to the same phase in the
>>> lifecycle, they execute in the order specificied in the POM.
>>> Beware this is the behavior I've observed and can't confirm it is
>>> specified
>>> behavior.
>>>
>>>
>>>
>>> João Cabrita
>>>
>>> On 16 February 2017 at 10:06, Paul Hammant <pa...@hammant.org> wrote:
>>>
>>> Hi folks,.
>>>>
>>>> I've a fast WebDriver using build that I blogged about: A 16 Second
>>>> Java
>>>> Webapp Build (Including WebDriver Tests)
>>>> <http://paulhammant.com/2017/02/05/a-16-second-java-webapp-
>>>> build-including-webdriver-tests/>
>>>> .
>>>>
>>>> Jooby (like SpringBoot and SparkJava) give new options for testing -
>>>> it
>>>> can
>>>> be instantiated in a JUnit test. Everything can be done in Surefire
>>>> now,
>>>> and the Failsafe plugin isn't needed for these. Don't believe me -
>>>> watch
>>>> the video in the blog entry above, it's not long.
>>>>
>>>> New problem. I want unit tests to run in this order:
>>>>
>>>> 1. unit
>>>> 2. integration (may invoke service calls headlessly)
>>>> 3. function (will use WebDriver)
>>>>
>>>>
>>>> I can't work out what magic I have to do with executions to allow
>>>> that to
>>>> happen.
>>>>
>>>> Here is how far I got:
>>>>
>>>>
>>>> https://github.com/paul-hammant/todobackend-jooby/blob/
>>>> master/pom.xml#L74
>>>>
>>>> It is all a bit second class, because I'd have to do ...
>>>>
>>>> mvn clean test -Punit-tests
>>>> mvn test -Pintegration-tests
>>>> mvn test -Pfunctional-tests
>>>>
>>>> ... to simulate a pipeline, and I would be happy to just rely on
>>>> annotations for classifications.
>>>>
>>>> I really want to do ...
>>>>
>>>> mvn clean test -DexecutionOrder=unit,integration,functional
>>>> -DstopBuildAtExecutionBoundariesForTestFailures
>>>>
>>>>
>>>> ... and shave seconds off the build.
>>>>
>>>> How do I configure that tersely and elegantly in the surefire plugin
>>>> today?
>>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
>> For additional commands, e-mail: users-help@maven.apache.org
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Configuring the surefire plugin - a question
Posted by Paul Hammant <pa...@hammant.org>.
Thanks João, thanks Robert.
I've taken Robert's snippet and expanded it a little to do what I want:
Diff:
https://github.com/paul-hammant/todobackend-jooby/commit/9626a3155eddbaea74bbf66a3e899b81227842ee
(repo: paul-hammant/todobackend-jooby* branch: expectations*)
I found that I had to be explicit about excludes too, and have a precursor
exclude that is outside of the three executions.
I did a bunch of trial and error, but this was the minima.
The test of correctness:
$ mvn install | grep "Time elapsed"
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.237 sec
- in todobackend.TodoUnitTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.51 sec
- in todobackend.TodoIntegrationTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 11.608
sec - in todobackend.TodoWebDriverTest
(ignore some WebDriver noise to std-err)
I can't say I really understand the rules about additive includes and
excludes, but my job is to make speedy builds, not fully understand every
angle bracket of Maven.
Regards,
- Paul
On Fri, Feb 17, 2017 at 2:00 PM, Robert Scholte <rf...@apache.org>
wrote:
> When you only want to change the pom (not the tests), a set of
> executionblocks will do the trick:
>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-surefire-plugin</artifactId>
> <version>2.18.1</version>
> <executions>
> <execution>
> <id>unit-tests</id>
> <configuration>
> <includes>
> <include>**/*Unit*.java</include>
> </includes>
> </configuration>
> </execution>
> <execution>
> <id>functional-tests</id>
> <configuration>
> <includes>
> <include>**/*WebDriver*.java</include>
> </includes>
> </configuration>
> </execution>
> <execution>
> <id>integration-tests</id>
> <configuration>
> <includes>
> <include>**/*Integration*.java</include>
> </includes>
> </configuration>
> </execution>
> <executions>
> </plugin>
>
>
> On Thu, 16 Feb 2017 11:22:37 +0100, João Cabrita <jo...@gmail.com>
> wrote:
>
> I'd say you could add executions to the surefire plugin with different
>> categories:
>> https://maven.apache.org/surefire/maven-surefire-plugin/
>> examples/junit.html#Using_JUnit_Categories
>>
>> Look at this gist (I've omitted some details):
>> https://gist.github.com/kewne/2b909ab5e8035a4e44e406fa35e3276c
>>
>> AFAIK, even if the executions are all bound to the same phase in the
>> lifecycle, they execute in the order specificied in the POM.
>> Beware this is the behavior I've observed and can't confirm it is
>> specified
>> behavior.
>>
>>
>>
>> João Cabrita
>>
>> On 16 February 2017 at 10:06, Paul Hammant <pa...@hammant.org> wrote:
>>
>> Hi folks,.
>>>
>>> I've a fast WebDriver using build that I blogged about: A 16 Second Java
>>> Webapp Build (Including WebDriver Tests)
>>> <http://paulhammant.com/2017/02/05/a-16-second-java-webapp-
>>> build-including-webdriver-tests/>
>>> .
>>>
>>> Jooby (like SpringBoot and SparkJava) give new options for testing - it
>>> can
>>> be instantiated in a JUnit test. Everything can be done in Surefire now,
>>> and the Failsafe plugin isn't needed for these. Don't believe me - watch
>>> the video in the blog entry above, it's not long.
>>>
>>> New problem. I want unit tests to run in this order:
>>>
>>> 1. unit
>>> 2. integration (may invoke service calls headlessly)
>>> 3. function (will use WebDriver)
>>>
>>>
>>> I can't work out what magic I have to do with executions to allow that to
>>> happen.
>>>
>>> Here is how far I got:
>>>
>>>
>>> https://github.com/paul-hammant/todobackend-jooby/blob/
>>> master/pom.xml#L74
>>>
>>> It is all a bit second class, because I'd have to do ...
>>>
>>> mvn clean test -Punit-tests
>>> mvn test -Pintegration-tests
>>> mvn test -Pfunctional-tests
>>>
>>> ... to simulate a pipeline, and I would be happy to just rely on
>>> annotations for classifications.
>>>
>>> I really want to do ...
>>>
>>> mvn clean test -DexecutionOrder=unit,integration,functional
>>> -DstopBuildAtExecutionBoundariesForTestFailures
>>>
>>>
>>> ... and shave seconds off the build.
>>>
>>> How do I configure that tersely and elegantly in the surefire plugin
>>> today?
>>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
Re: Configuring the surefire plugin - a question
Posted by Robert Scholte <rf...@apache.org>.
When you only want to change the pom (not the tests), a set of
executionblocks will do the trick:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<executions>
<execution>
<id>unit-tests</id>
<configuration>
<includes>
<include>**/*Unit*.java</include>
</includes>
</configuration>
</execution>
<execution>
<id>functional-tests</id>
<configuration>
<includes>
<include>**/*WebDriver*.java</include>
</includes>
</configuration>
</execution>
<execution>
<id>integration-tests</id>
<configuration>
<includes>
<include>**/*Integration*.java</include>
</includes>
</configuration>
</execution>
<executions>
</plugin>
On Thu, 16 Feb 2017 11:22:37 +0100, João Cabrita
<jo...@gmail.com> wrote:
> I'd say you could add executions to the surefire plugin with different
> categories:
> https://maven.apache.org/surefire/maven-surefire-plugin/examples/junit.html#Using_JUnit_Categories
>
> Look at this gist (I've omitted some details):
> https://gist.github.com/kewne/2b909ab5e8035a4e44e406fa35e3276c
>
> AFAIK, even if the executions are all bound to the same phase in the
> lifecycle, they execute in the order specificied in the POM.
> Beware this is the behavior I've observed and can't confirm it is
> specified
> behavior.
>
>
>
> João Cabrita
>
> On 16 February 2017 at 10:06, Paul Hammant <pa...@hammant.org> wrote:
>
>> Hi folks,.
>>
>> I've a fast WebDriver using build that I blogged about: A 16 Second Java
>> Webapp Build (Including WebDriver Tests)
>> <http://paulhammant.com/2017/02/05/a-16-second-java-webapp-
>> build-including-webdriver-tests/>
>> .
>>
>> Jooby (like SpringBoot and SparkJava) give new options for testing - it
>> can
>> be instantiated in a JUnit test. Everything can be done in Surefire now,
>> and the Failsafe plugin isn't needed for these. Don't believe me -
>> watch
>> the video in the blog entry above, it's not long.
>>
>> New problem. I want unit tests to run in this order:
>>
>> 1. unit
>> 2. integration (may invoke service calls headlessly)
>> 3. function (will use WebDriver)
>>
>>
>> I can't work out what magic I have to do with executions to allow that
>> to
>> happen.
>>
>> Here is how far I got:
>>
>>
>> https://github.com/paul-hammant/todobackend-jooby/blob/master/pom.xml#L74
>>
>> It is all a bit second class, because I'd have to do ...
>>
>> mvn clean test -Punit-tests
>> mvn test -Pintegration-tests
>> mvn test -Pfunctional-tests
>>
>> ... to simulate a pipeline, and I would be happy to just rely on
>> annotations for classifications.
>>
>> I really want to do ...
>>
>> mvn clean test -DexecutionOrder=unit,integration,functional
>> -DstopBuildAtExecutionBoundariesForTestFailures
>>
>>
>> ... and shave seconds off the build.
>>
>> How do I configure that tersely and elegantly in the surefire plugin
>> today?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Configuring the surefire plugin - a question
Posted by João Cabrita <jo...@gmail.com>.
I'd say you could add executions to the surefire plugin with different
categories:
https://maven.apache.org/surefire/maven-surefire-plugin/examples/junit.html#Using_JUnit_Categories
Look at this gist (I've omitted some details):
https://gist.github.com/kewne/2b909ab5e8035a4e44e406fa35e3276c
AFAIK, even if the executions are all bound to the same phase in the
lifecycle, they execute in the order specificied in the POM.
Beware this is the behavior I've observed and can't confirm it is specified
behavior.
João Cabrita
On 16 February 2017 at 10:06, Paul Hammant <pa...@hammant.org> wrote:
> Hi folks,.
>
> I've a fast WebDriver using build that I blogged about: A 16 Second Java
> Webapp Build (Including WebDriver Tests)
> <http://paulhammant.com/2017/02/05/a-16-second-java-webapp-
> build-including-webdriver-tests/>
> .
>
> Jooby (like SpringBoot and SparkJava) give new options for testing - it can
> be instantiated in a JUnit test. Everything can be done in Surefire now,
> and the Failsafe plugin isn't needed for these. Don't believe me - watch
> the video in the blog entry above, it's not long.
>
> New problem. I want unit tests to run in this order:
>
> 1. unit
> 2. integration (may invoke service calls headlessly)
> 3. function (will use WebDriver)
>
>
> I can't work out what magic I have to do with executions to allow that to
> happen.
>
> Here is how far I got:
>
>
> https://github.com/paul-hammant/todobackend-jooby/blob/master/pom.xml#L74
>
> It is all a bit second class, because I'd have to do ...
>
> mvn clean test -Punit-tests
> mvn test -Pintegration-tests
> mvn test -Pfunctional-tests
>
> ... to simulate a pipeline, and I would be happy to just rely on
> annotations for classifications.
>
> I really want to do ...
>
> mvn clean test -DexecutionOrder=unit,integration,functional
> -DstopBuildAtExecutionBoundariesForTestFailures
>
>
> ... and shave seconds off the build.
>
> How do I configure that tersely and elegantly in the surefire plugin today?
>