[jira] [Commented] (SUREFIRE-1728) maven.test.failure.ignore: differentiate between test failure and timeout

["Falko Modler (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SUREFIRE-1728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16985582#comment-16985582 ] 

Falko Modler commented on SUREFIRE-1728:
----------------------------------------

[~tibordigana] There seems to be a misunderstanding: This issue is _not_ about the wording of the message (that you have just modified in the linked commit)!
Instead we'd need a mode in surefire which causes a "hard error" in case of a timeout when {{maven.test.failure.ignore}} is activated, but without changing the behaviour for failing tests.

Therefore I don't see how this is fixed. This ticket should be reopened.

> maven.test.failure.ignore: differentiate between test failure and timeout
> -------------------------------------------------------------------------
>
>                 Key: SUREFIRE-1728
>                 URL: https://issues.apache.org/jira/browse/SUREFIRE-1728
>             Project: Maven Surefire
>          Issue Type: Improvement
>    Affects Versions: 2.22.2, 3.0.0-M3
>         Environment: Maven 3.6.2
>            Reporter: Falko Modler
>            Assignee: Tibor Digana
>            Priority: Major
>             Fix For: 3.0.0-M5
>
>
> On a build server like Jenkins, people typically set {{-Dmaven.test.failure.ignore=true}} to get the maximum number of test results instead of failing the build after the first module with test failures.
> Unfortunately, timeouts are also ignored when this property is activated, leaving the Jenkins JUnit plugin no chance to detect that something went wrong (because a timeout is not reported in a txt or xml report).
> See also [JENKINS-46553|https://issues.jenkins-ci.org/browse/JENKINS-46553].
> The two cases should be differentiated.
> Due to backward compatibility reasons, I am not sure whether it would be wise to simply exclude timeout cases.
> One backward compatible solution might be to extend the value range of {{maven.test.failure.ignore}} from just {{true}} XOR {{false}} to something like:
> {{true}}/{{all}} XOR {{failure}} XOR {{false}}.
> The alternative would be to introduce yet another property...



--
This message was sent by Atlassian Jira
(v8.3.4#803005)