You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by "Walton, Mark (CGI Federal)" <ma...@cgifederal.com> on 2018/04/23 19:22:02 UTC
Skipped tests
Hi:
Does anyone know how to configure SureFire so that skipped tests result in Build Failure
Thanks
Proprietary/confidential information belonging to CGI Federal Inc. or its affiliates may be contained in this message. If you are not a recipient indicated or intended in this message (or responsible for the delivery of this message to such person), or if you think for any reason that this message may have been addressed to you in error, you may not use or copy or deliver this message to anyone else. In such case, you should destroy this message and are asked to notify the sender by reply email.
RE: Skipped tests
Posted by "Walton, Mark (CGI Federal)" <ma...@cgifederal.com>.
Hi:
No I do not. In the following output I want "Build Success" to be replaced by "Build Failure" because Skipped = 10.
Tests run: 14880, Failures: 0, Errors: 0, Skipped: 10
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 43.306 s
[INFO] Finished at: 2018-04-23T14:43:38-06:00
[INFO] Final Memory: 52M/698M
[INFO] -------------------------------------------------------------------
Thanks
Mark
-----Original Message-----
From: Tibor Digana [mailto:tibordigana@apache.org]
Sent: Monday, April 23, 2018 3:58 PM
To: Maven Developers List
Subject: Re: Skipped tests
Do you want to turn the JUnit annotation @Ignore to a failure in plugin/build summary?
I think there is no such possibility.
On Mon, Apr 23, 2018 at 9:22 PM, Walton, Mark (CGI Federal) < mark.walton@cgifederal.com> wrote:
> Hi:
> Does anyone know how to configure SureFire so that skipped tests
> result in Build Failure
>
> Thanks
> Proprietary/confidential information belonging to CGI Federal Inc. or
> its affiliates may be contained in this message. If you are not a
> recipient indicated or intended in this message (or responsible for
> the delivery of this message to such person), or if you think for any
> reason that this message may have been addressed to you in error, you
> may not use or copy or deliver this message to anyone else. In such
> case, you should destroy this message and are asked to notify the sender by reply email.
>
Proprietary/confidential information belonging to CGI Federal Inc. or its affiliates may be contained in this message. If you are not a recipient indicated or intended in this message (or responsible for the delivery of this message to such person), or if you think for any reason that this message may have been addressed to you in error, you may not use or copy or deliver this message to anyone else. In such case, you should destroy this message and are asked to notify the sender by reply email.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Skipped tests
Posted by Tibor Digana <ti...@apache.org>.
Do you want to turn the JUnit annotation @Ignore to a failure in
plugin/build summary?
I think there is no such possibility.
On Mon, Apr 23, 2018 at 9:22 PM, Walton, Mark (CGI Federal) <
mark.walton@cgifederal.com> wrote:
> Hi:
> Does anyone know how to configure SureFire so that skipped tests result in
> Build Failure
>
> Thanks
> Proprietary/confidential information belonging to CGI Federal Inc. or its
> affiliates may be contained in this message. If you are not a recipient
> indicated or intended in this message (or responsible for the delivery of
> this message to such person), or if you think for any reason that this
> message may have been addressed to you in error, you may not use or copy or
> deliver this message to anyone else. In such case, you should destroy this
> message and are asked to notify the sender by reply email.
>
Re: Skipped tests
Posted by Stephen Connolly <st...@gmail.com>.
On Mon 23 Apr 2018 at 21:52, Walton, Mark (CGI Federal) <
mark.walton@cgifederal.com> wrote:
> Hi Marco:
> I tried what you suggested. As you can see from the output below I have 10
> Skipped but I am still getting Build Success. Any time that Skipped is >0 I
> want Build Failure.
So to be clear on your ask: if you have a @Theory or parameterised test
which excludes some data point combinations using the *by design*
assumeThat because the test does not make sense for that combination, you
want the build to fail?
In my opinion such a configuration would be excluding some of the
exceedingly useful functionality of JUnit and if you did that on a project
I was working on I would feel well within my rights to complain at you
vociferously every time I saw you.
I respectfully advise that you take a step back and consider what skipped
is supposed to be used for. I suspect the developers on your team are using
it incorrectly. Code review needs to cover tests. If a test must not be
ignored, code review must ensure it does not skip. Skipped test cases are
necessary for parameterised tests with combinations that cannot pass the
test assertions due to their nature.
@Parameterized
public void nonEquality(Widget x, Widget y) {
assumeThat(x, not(sameInstance(y));
assertThat(x, not(is(y)));
}
Will always skip the case where the two parameters are the same instance,
where equality would be true.
You need skipped for such tests
>
> Tests run: 14880, Failures: 0, Errors: 0, Skipped: 10
>
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 43.306 s
> [INFO] Finished at: 2018-04-23T14:43:38-06:00
> [INFO] Final Memory: 52M/698M
> [INFO] -------------------------------------------------------------------
>
> Thanks
> Mark Walton
>
> -----Original Message-----
> From: Marco Schulz [mailto:marco.schulz@outlook.com]
> Sent: Monday, April 23, 2018 2:26 PM
> To: dev@maven.apache.org
> Subject: Re: Skipped tests
>
> hi
>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-surefire-plugin</artifactId>
> <version>2.21.0</version>
> <configuration>
> <argLine>-XX:-UseSplitVerifier</argLine>
> <testFailureIgnore>true</testFailureIgnore>
> <skipTests>${skipTests}</skipTests>
> </configuration>
> </plugin>
>
>
> see the full POM at:
> https://github.com/ElmarDott/TP-CM/blob/master/build-workflow/pom.xml
>
> On Mon, 2018-04-23 at 19:22 +0000, Walton, Mark (CGI Federal) wrote:
> Hi:
> Does anyone know how to configure SureFire so that skipped tests result in
> Build Failure
>
> Thanks
> Proprietary/confidential information belonging to CGI Federal Inc. or its
> affiliates may be contained in this message. If you are not a recipient
> indicated or intended in this message (or responsible for the delivery of
> this message to such person), or if you think for any reason that this
> message may have been addressed to you in error, you may not use or copy or
> deliver this message to anyone else. In such case, you should destroy this
> message and are asked to notify the sender by reply email.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
--
Sent from my phone
RE: Skipped tests
Posted by "Walton, Mark (CGI Federal)" <ma...@cgifederal.com>.
Hi Marco:
I tried what you suggested. As you can see from the output below I have 10 Skipped but I am still getting Build Success. Any time that Skipped is >0 I want Build Failure.
Tests run: 14880, Failures: 0, Errors: 0, Skipped: 10
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 43.306 s
[INFO] Finished at: 2018-04-23T14:43:38-06:00
[INFO] Final Memory: 52M/698M
[INFO] -------------------------------------------------------------------
Thanks
Mark Walton
-----Original Message-----
From: Marco Schulz [mailto:marco.schulz@outlook.com]
Sent: Monday, April 23, 2018 2:26 PM
To: dev@maven.apache.org
Subject: Re: Skipped tests
hi
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.21.0</version>
<configuration>
<argLine>-XX:-UseSplitVerifier</argLine>
<testFailureIgnore>true</testFailureIgnore>
<skipTests>${skipTests}</skipTests>
</configuration>
</plugin>
see the full POM at:
https://github.com/ElmarDott/TP-CM/blob/master/build-workflow/pom.xml
On Mon, 2018-04-23 at 19:22 +0000, Walton, Mark (CGI Federal) wrote:
Hi:
Does anyone know how to configure SureFire so that skipped tests result in Build Failure
Thanks
Proprietary/confidential information belonging to CGI Federal Inc. or its affiliates may be contained in this message. If you are not a recipient indicated or intended in this message (or responsible for the delivery of this message to such person), or if you think for any reason that this message may have been addressed to you in error, you may not use or copy or deliver this message to anyone else. In such case, you should destroy this message and are asked to notify the sender by reply email.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Skipped tests
Posted by Marco Schulz <ma...@outlook.com>.
hi
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.21.0</version>
<configuration>
<argLine>-XX:-UseSplitVerifier</argLine>
<testFailureIgnore>true</testFailureIgnore>
<skipTests>${skipTests}</skipTests>
</configuration>
</plugin>
see the full POM at:
https://github.com/ElmarDott/TP-CM/blob/master/build-workflow/pom.xml
On Mon, 2018-04-23 at 19:22 +0000, Walton, Mark (CGI Federal) wrote:
Hi:
Does anyone know how to configure SureFire so that skipped tests result in Build Failure
Thanks
Proprietary/confidential information belonging to CGI Federal Inc. or its affiliates may be contained in this message. If you are not a recipient indicated or intended in this message (or responsible for the delivery of this message to such person), or if you think for any reason that this message may have been addressed to you in error, you may not use or copy or deliver this message to anyone else. In such case, you should destroy this message and are asked to notify the sender by reply email.