You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Christian Stein (Jira)" <ji...@apache.org> on 2020/02/10 15:56:00 UTC
[jira] [Commented] (SUREFIRE-1688) Junit Platform test with failing
@BeforeAll does not fail the build
[ https://issues.apache.org/jira/browse/SUREFIRE-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17033712#comment-17033712 ]
Christian Stein commented on SUREFIRE-1688:
-------------------------------------------
{quote}[DEBUG] -- end configuration --
[DEBUG] Surefire report directory: /home/hheg/git/junit_fail/target/surefire-reports
[DEBUG] Setting system property [user.dir]=[/home/hheg/git/junit_fail]
[DEBUG] Setting system property [localRepository]=[/home/hheg/.m2/repository]
[DEBUG] Setting system property [basedir]=[/home/hheg/git/junit_fail]
[DEBUG] Using JVM: /usr/lib/jvm/adoptopenjdk-8-hotspot-amd64/jre/bin/java with Java version 1.8
[DEBUG] Adding to surefire booter test classpath: /home/hheg/.m2/repository/org/apache/maven/surefire/surefire-booter/3.0.0-M4/surefire-booter-3.0.0-M4.jar Scope: compile
[DEBUG] Adding to surefire booter test classpath: /home/hheg/.m2/repository/org/apache/maven/surefire/surefire-api/3.0.0-M4/surefire-api-3.0.0-M4.jar Scope: compile
[DEBUG] Adding to surefire booter test classpath: /home/hheg/.m2/repository/org/apache/maven/surefire/surefire-logger-api/3.0.0-M4/surefire-logger-api-3.0.0-M4.jar Scope: compile
[DEBUG] Adding to surefire test classpath: /home/hheg/.m2/repository/org/apache/maven/surefire/surefire-junit-platform/3.0.0-M4/surefire-junit-platform-3.0.0-M4.jar Scope: null
[DEBUG] Adding to surefire test classpath: /home/hheg/.m2/repository/org/apache/maven/surefire/common-java5/3.0.0-M4/common-java5-3.0.0-M4.jar Scope: compile
[DEBUG] Adding to surefire test classpath: /home/hheg/.m2/repository/org/apache/maven/surefire/surefire-api/3.0.0-M4/surefire-api-3.0.0-M4.jar Scope: compile
[DEBUG] Adding to surefire test classpath: /home/hheg/.m2/repository/org/apache/maven/surefire/surefire-logger-api/3.0.0-M4/surefire-logger-api-3.0.0-M4.jar Scope: compile
[DEBUG] Adding to surefire test classpath: /home/hheg/.m2/repository/org/junit/platform/junit-platform-launcher/1.3.2/junit-platform-launcher-1.3.2.jar Scope: compile
[DEBUG] Adding to surefire test classpath: /home/hheg/.m2/repository/org/apiguardian/apiguardian-api/1.0.0/apiguardian-api-1.0.0.jar Scope: compile
[DEBUG] Adding to surefire test classpath: /home/hheg/.m2/repository/org/junit/platform/junit-platform-engine/1.3.2/junit-platform-engine-1.3.2.jar Scope: compile
[DEBUG] Adding to surefire test classpath: /home/hheg/.m2/repository/org/junit/platform/junit-platform-commons/1.3.2/junit-platform-commons-1.3.2.jar Scope: compile
[DEBUG] Adding to surefire test classpath: /home/hheg/.m2/repository/org/opentest4j/opentest4j/1.1.1/opentest4j-1.1.1.jar Scope: compile
[DEBUG] Test dependencies contain org.junit.platform:junit-platform-engine
[DEBUG] Resolving artifact org.junit.platform:junit-platform-engine:1.6.0
[DEBUG] Resolved artifact org.junit.platform:junit-platform-engine:1.6.0 to [org.junit.platform:junit-platform-engine:jar:1.6.0, org.apiguardian:apiguardian-api:jar:1.1.0:compile, org.opentest4j:opentest4j:jar:1.2.0:compile, org.junit.platform:junit-platform-commons:jar:1.6.0:compile]
[DEBUG] Removed artifact org.junit.platform:junit-platform-engine:jar:1.6.0 from provider. Already appears in test classpath.
[DEBUG] Removed artifact org.apiguardian:apiguardian-api:jar:1.1.0:compile from provider. Already appears in test classpath.
[DEBUG] Removed artifact org.opentest4j:opentest4j:jar:1.2.0:compile from provider. Already appears in test classpath.
[DEBUG] Removed artifact org.junit.platform:junit-platform-commons:jar:1.6.0:compile from provider. Already appears in test classpath.
[DEBUG] Resolving artifact org.junit.platform:junit-platform-launcher:1.6.0
[DEBUG] Resolved artifact org.junit.platform:junit-platform-launcher:1.6.0 to [org.junit.platform:junit-platform-launcher:jar:1.6.0, org.apiguardian:apiguardian-api:jar:1.1.0:compile, org.junit.platform:junit-platform-engine:jar:1.6.0:compile, org.opentest4j:opentest4j:jar:1.2.0:compile, org.junit.platform:junit-platform-commons:jar:1.6.0:compile]
[DEBUG] test classpath: /home/hheg/git/junit_fail/target/test-classes /home/hheg/git/junit_fail/target/classes /home/hheg/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.6.0/junit-jupiter-api-5.6.0.jar /home/hheg/.m2/repository/org/apiguardian/apiguardian-api/1.1.0/apiguardian-api-1.1.0.jar /home/hheg/.m2/repository/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar /home/hheg/.m2/repository/org/junit/platform/junit-platform-commons/1.6.0/junit-platform-commons-1.6.0.jar /home/hheg/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.6.0/junit-jupiter-engine-5.6.0.jar /home/hheg/.m2/repository/org/junit/platform/junit-platform-engine/1.6.0/junit-platform-engine-1.6.0.jar /home/hheg/.m2/repository/org/junit/platform/junit-platform-launcher/1.6.0/junit-platform-launcher-1.6.0.jar
[DEBUG] provider classpath: /home/hheg/.m2/repository/org/apache/maven/surefire/surefire-junit-platform/3.0.0-M4/surefire-junit-platform-3.0.0-M4.jar /home/hheg/.m2/repository/org/apache/maven/surefire/surefire-api/3.0.0-M4/surefire-api-3.0.0-M4.jar /home/hheg/.m2/repository/org/apache/maven/surefire/surefire-logger-api/3.0.0-M4/surefire-logger-api-3.0.0-M4.jar /home/hheg/.m2/repository/org/apache/maven/surefire/common-java5/3.0.0-M4/common-java5-3.0.0-M4.jar
[DEBUG] test(compact) classpath: test-classes classes junit-jupiter-api-5.6.0.jar apiguardian-api-1.1.0.jar opentest4j-1.2.0.jar junit-platform-commons-1.6.0.jar junit-jupiter-engine-5.6.0.jar junit-platform-engine-1.6.0.jar junit-platform-launcher-1.6.0.jar
[DEBUG] provider(compact) classpath: surefire-junit-platform-3.0.0-M4.jar surefire-api-3.0.0-M4.jar surefire-logger-api-3.0.0-M4.jar common-java5-3.0.0-M4.jar
[DEBUG] in-process classpath: /home/hheg/.m2/repository/org/apache/maven/surefire/surefire-junit-platform/3.0.0-M4/surefire-junit-platform-3.0.0-M4.jar /home/hheg/.m2/repository/org/apache/maven/surefire/surefire-api/3.0.0-M4/surefire-api-3.0.0-M4.jar /home/hheg/.m2/repository/org/apache/maven/surefire/surefire-logger-api/3.0.0-M4/surefire-logger-api-3.0.0-M4.jar /home/hheg/.m2/repository/org/apache/maven/surefire/common-java5/3.0.0-M4/common-java5-3.0.0-M4.jar /home/hheg/.m2/repository/org/apache/maven/surefire/maven-surefire-common/3.0.0-M4/maven-surefire-common-3.0.0-M4.jar /home/hheg/.m2/repository/org/apache/maven/surefire/surefire-extensions-api/3.0.0-M4/surefire-extensions-api-3.0.0-M4.jar
[DEBUG] in-process(compact) classpath: surefire-junit-platform-3.0.0-M4.jar surefire-api-3.0.0-M4.jar surefire-logger-api-3.0.0-M4.jar common-java5-3.0.0-M4.jar maven-surefire-common-3.0.0-M4.jar surefire-extensions-api-3.0.0-M4.jar
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[DEBUG] Determined Maven Process ID 12066
[DEBUG] boot classpath: /home/hheg/.m2/repository/org/apache/maven/surefire/surefire-booter/3.0.0-M4/surefire-booter-3.0.0-M4.jar /home/hheg/.m2/repository/org/apache/maven/surefire/surefire-api/3.0.0-M4/surefire-api-3.0.0-M4.jar /home/hheg/.m2/repository/org/apache/maven/surefire/surefire-logger-api/3.0.0-M4/surefire-logger-api-3.0.0-M4.jar /home/hheg/git/junit_fail/target/test-classes /home/hheg/git/junit_fail/target/classes /home/hheg/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.6.0/junit-jupiter-api-5.6.0.jar /home/hheg/.m2/repository/org/apiguardian/apiguardian-api/1.1.0/apiguardian-api-1.1.0.jar /home/hheg/.m2/repository/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar /home/hheg/.m2/repository/org/junit/platform/junit-platform-commons/1.6.0/junit-platform-commons-1.6.0.jar /home/hheg/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.6.0/junit-jupiter-engine-5.6.0.jar /home/hheg/.m2/repository/org/junit/platform/junit-platform-engine/1.6.0/junit-platform-engine-1.6.0.jar /home/hheg/.m2/repository/org/junit/platform/junit-platform-launcher/1.6.0/junit-platform-launcher-1.6.0.jar /home/hheg/.m2/repository/org/apache/maven/surefire/surefire-junit-platform/3.0.0-M4/surefire-junit-platform-3.0.0-M4.jar /home/hheg/.m2/repository/org/apache/maven/surefire/common-java5/3.0.0-M4/common-java5-3.0.0-M4.jar
[DEBUG] boot(compact) classpath: surefire-booter-3.0.0-M4.jar surefire-api-3.0.0-M4.jar surefire-logger-api-3.0.0-M4.jar test-classes classes junit-jupiter-api-5.6.0.jar apiguardian-api-1.1.0.jar opentest4j-1.2.0.jar junit-platform-commons-1.6.0.jar junit-jupiter-engine-5.6.0.jar junit-platform-engine-1.6.0.jar junit-platform-launcher-1.6.0.jar surefire-junit-platform-3.0.0-M4.jar common-java5-3.0.0-M4.jar
[DEBUG] Forking command line: /bin/sh -c cd /home/hheg/git/junit_fail && /usr/lib/jvm/adoptopenjdk-8-hotspot-amd64/jre/bin/java -jar /home/hheg/git/junit_fail/target/surefire/surefirebooter1335223135765781073.jar /home/hheg/git/junit_fail/target/surefire 2020-02-10T14-11-42_186-jvmRun1 surefire2241776508346597516tmp surefire_01062025114292542072tmp
[DEBUG] PpidChecker\{ppid=12066, stopped=false, canExecuteLocalUnixPs=false, canExecuteStandardUnixPs=true}
[INFO] Running junit_fail.JunitFailTest
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.015 s - in junit_fail.JunitFailTest
{quote}
Seems like Surefire 3.0.0-M4 loaded JUnit Platform artifacts version 1.3.2 into it's classpath. That seems to conflict with the version 1.6.0 the user requested by using JUnit Jupiter 5.6.0
See also: https://github.com/junit-team/junit5/issues/2178
> Junit Platform test with failing @BeforeAll does not fail the build
> -------------------------------------------------------------------
>
> Key: SUREFIRE-1688
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1688
> Project: Maven Surefire
> Issue Type: Bug
> Components: JUnit 5.x support, Maven Surefire Plugin
> Reporter: Guy Brand
> Assignee: Tibor Digana
> Priority: Major
> Fix For: 3.0.0-M4
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When using Surefire {{3.0.0-SNAPSHOT}} and JUnit Platform {{5.4.2}} the following test does not fail the Maven build.
> {code:java}
> import org.junit.jupiter.api.Test;
> import org.junit.jupiter.api.BeforeAll;
> import org.junit.jupiter.api.Assertions;
> public class Tests {
> @BeforeAll
> public static void beforeAll() { Assertions.fail("BeforeAll fail"); }
> @Test
> public void test() { Assertions.fail("Test fail"); }
> }
> {code}
> And its output:
> {code}
> [INFO] --- maven-surefire-plugin:3.0.0-SNAPSHOT:test (default-test) @ top-level-project ---
> [INFO]
> [INFO] -------------------------------------------------------
> [INFO] T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running Tests
> [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.015 s - in Tests
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> {code}
> When removing the {{@BeforeAll}} annotation/method it fails as expected.
> {{3.0.0-M3}} version of Surefire works as expected:
> {code}
> [INFO] --- maven-surefire-plugin:3.0.0-M3:test (default-test) @ top-level-project ---
> [INFO]
> [INFO] -------------------------------------------------------
> [INFO] T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running Tests
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.008 s <<< FAILURE! - in Tests
> [ERROR] Tests Time elapsed: 0.007 s <<< FAILURE!
> org.opentest4j.AssertionFailedError: BeforeAll fail
> at Tests.beforeAll(Tests.java:7)
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Failures:
> [ERROR] Tests.beforeAll:7 BeforeAll fail
> [INFO]
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
> [INFO]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 2.373 s
> [INFO] Finished at: 2019-08-20T16:20:16+02:00
> [INFO] ------------------------------------------------------------------------
> [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M3:test (default-test) on project top-level-project: There are test failures.
> {code}
> For all the tests Maven 3.6.1 has been used.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)