[jira] [Created] (SUREFIRE-1688) Junit Platform test with failing @BeforeAll does not fail the build

["Guy Brand (Jira)" <ji...@apache.org>]

Guy Brand created SUREFIRE-1688:
-----------------------------------

             Summary: Junit Platform test with failing @BeforeAll does not fail the build
                 Key: SUREFIRE-1688
                 URL: https://issues.apache.org/jira/browse/SUREFIRE-1688
             Project: Maven Surefire
          Issue Type: Bug
          Components: JUnit 5.x support, Maven Surefire Plugin
            Reporter: Guy Brand


When using Surefire {{3.0.0-SNAPSHOT}} and JUnit Platform {{5.4.2}} the following test does not fail the Maven build. 

{code:java}
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Assertions;
public class Tests {

    @BeforeAll
    public static void beforeAll() { Assertions.fail("BeforeAll fail"); }

    @Test
    public void test() { Assertions.fail("Test fail"); }
}
{code}

And its output:

{code}
[INFO] --- maven-surefire-plugin:3.0.0-SNAPSHOT:test (default-test) @ top-level-project ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running Tests
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.015 s - in Tests
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
{code}

When removing the {{@BeforeAll}} annotation/method it fails as expected. 

{{3.0.0-M3}} version of Surefire works as expected:

{code}
[INFO] --- maven-surefire-plugin:3.0.0-M3:test (default-test) @ top-level-project ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running Tests
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.008 s <<< FAILURE! - in Tests
[ERROR] Tests  Time elapsed: 0.007 s  <<< FAILURE!
org.opentest4j.AssertionFailedError: BeforeAll fail
	at Tests.beforeAll(Tests.java:7)

[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   Tests.beforeAll:7 BeforeAll fail
[INFO] 
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  2.373 s
[INFO] Finished at: 2019-08-20T16:20:16+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M3:test (default-test) on project top-level-project: There are test failures.
{code}

For all the tests Maven 3.6.1 has been used.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)