[jira] [Closed] (SUREFIRE-1799) Exception thrown in @MethodSource method results in "no tests"

["Tibor Digana (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SUREFIRE-1799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tibor Digana closed SUREFIRE-1799.
----------------------------------
      Assignee: Tibor Digana
    Resolution: Cannot Reproduce

Fixed in 3.0.0-M5.

> Exception thrown in @MethodSource method results in "no tests"
> --------------------------------------------------------------
>
>                 Key: SUREFIRE-1799
>                 URL: https://issues.apache.org/jira/browse/SUREFIRE-1799
>             Project: Maven Surefire
>          Issue Type: Bug
>          Components: JUnit 5.x support
>    Affects Versions: 3.0.0-M4
>            Reporter: Jens Reimann
>            Assignee: Tibor Digana
>            Priority: Major
>
> Running the following code:
> {code:java}
> import java.util.List;
> import org.junit.jupiter.api.TestInstance;
> import org.junit.jupiter.api.TestInstance.Lifecycle;
> import org.junit.jupiter.params.ParameterizedTest;
> import org.junit.jupiter.params.provider.MethodSource;
> @TestInstance(Lifecycle.PER_CLASS)
> public class MethodSourceErrorTest {
>     public List<String> getValues() {
>         throw new RuntimeException("Foo"); // fails
>         // return Arrays.asList("a"); // works
>     }
>     @ParameterizedTest
>     @MethodSource("getValues")
>     public void test1(String value) {
>         System.out.println(value);
>     }
> }
> {code}
> Results in:
> {code}
> [INFO] -------------------------------------------------------
> [INFO]  T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running foo.MethodSourceErrorTest
> [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.018 s - in foo.MethodSourceErrorTest
> [INFO] 
> [INFO] Results:
> [INFO] 
> [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
> [INFO] 
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> {code}
> My expectation would be that:
>  * JUnit reports an error for the tests using the method source
>  * The build fails
> Additional information:
>  * It fails the same way, being a static method
> Used versions (Jupiter/Vintage/Platform): 5.6.2, 1.6.2
> I raised an issue with JUnit 5 (https://github.com/junit-team/junit5/issues/2319), however they claim that it is an issue with the tool running the test (Maven Surefire in this case).
> And indeed the TestExecutionListener receives an exception:
> {code:java}
> public class MyTestExecutionListener implements TestExecutionListener {
>     public MyTestExecutionListener() {
>         System.out.println("Here");
>     }
>     @Override
>     public void executionFinished(TestIdentifier testIdentifier, TestExecutionResult testExecutionResult) {
>         System.out.format("finished - id: %s, result: %s%n", testIdentifier, testExecutionResult);
>     }
> }
> {code}
> Shows the exception as:
> {code}
> [INFO] --- maven-surefire-plugin:3.0.0-M4:test (default-test) @ bar ---
> [WARNING] useSystemClassloader setting has no effect when not forking
> Here
> [INFO] Running foo.MethodSourceErrorTest
> finished - id: TestIdentifier [uniqueId = '[engine:junit-jupiter]/[class:foo.MethodSourceErrorTest]/[test-template:test1(java.lang.String)]', parentId = '[engine:junit-jupiter]/[class:foo.MethodSourceErrorTest]', displayName = 'test1(String)', legacyReportingName = 'test1(String)', source = MethodSource [className = 'foo.MethodSourceErrorTest', methodName = 'test1', methodParameterTypes = 'java.lang.String'], tags = [], type = CONTAINER], result: TestExecutionResult [status = FAILED, throwable = java.lang.RuntimeException: Foo]
> finished - id: TestIdentifier [uniqueId = '[engine:junit-jupiter]/[class:foo.MethodSourceErrorTest]', parentId = '[engine:junit-jupiter]', displayName = 'MethodSourceErrorTest', legacyReportingName = 'foo.MethodSourceErrorTest', source = ClassSource [className = 'foo.MethodSourceErrorTest', filePosition = null], tags = [], type = CONTAINER], result: TestExecutionResult [status = SUCCESSFUL, throwable = null]
> [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.013 s - in foo.MethodSourceErrorTest
> finished - id: TestIdentifier [uniqueId = '[engine:junit-jupiter]', parentId = null, displayName = 'JUnit Jupiter', legacyReportingName = 'JUnit Jupiter', source = null, tags = [], type = CONTAINER], result: TestExecutionResult [status = SUCCESSFUL, throwable = null]
> [INFO] 
> [INFO] Results:
> [INFO] 
> [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)