You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Albert Johnston (Jira)" <ji...@apache.org> on 2019/12/27 23:00:03 UTC
[jira] [Updated] (SUREFIRE-1741) Exceptions in parameterized test
sources are ignored
[ https://issues.apache.org/jira/browse/SUREFIRE-1741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Albert Johnston updated SUREFIRE-1741:
--------------------------------------
Description:
Associated versions on my end at time of testing:
* JUnit Jupiter 5.5.1
* JDK 11.0.5
* Maven 3.6.1
If the following code is run, either for Surefire or Failsafe, only two of the test methods are logged as having been detected at all. The third one is completely ignored as if it didn't exist.
{code:java}
public class ExampleTest {
@Test
public void shouldRunAndPassAsExpected() {
}
public static Stream<Arguments> nothing() {
return Stream.of(Arguments.arguments());
}
@ParameterizedTest
@MethodSource("nothing")
public void shouldRunAndFailAsExpected() {
Assertions.fail();
}
static Stream<Arguments> throwException() {
if (true) throw new RuntimeException("Stop");
return Stream.of(Arguments.arguments());
}
@ParameterizedTest
@MethodSource("throwException")
void willNotBeLoggedAsFailingDespiteNotRunning() {
Assertions.fail();
}
}
{code}
This is particularly obvious if the first two test methods are commented out, as running mvn test will log that 0 tests were run.
In case it helps, here's the output I'm seeing when running it with the first two tests commented out:
{noformat}
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running ExampleIT
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.02 s - in ExampleIT
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0{noformat}
was:
Associated versions on my end at time of testing:
* JUnit Jupiter 5.5.1
* JDK 11.0.5
* Maven 3.6.1
Associated Java version at time of testing on my end: 11
If the following code is run, either for Surefire or Failsafe, only two of the test methods are logged as having been detected at all. The third one is completely ignored as if it didn't exist.
{code:java}
public class ExampleTest {
@Test
public void shouldRunAndPassAsExpected() {
}
public static Stream<Arguments> nothing() {
return Stream.of(Arguments.arguments());
}
@ParameterizedTest
@MethodSource("nothing")
public void shouldRunAndFailAsExpected() {
Assertions.fail();
}
static Stream<Arguments> throwException() {
if (true) throw new RuntimeException("Stop");
return Stream.of(Arguments.arguments());
}
@ParameterizedTest
@MethodSource("throwException")
void willNotBeLoggedAsFailingDespiteNotRunning() {
Assertions.fail();
}
}
{code}
This is particularly obvious if the first two test methods are commented out, as running mvn test will log that 0 tests were run.
In case it helps, here's the output I'm seeing when running it with the first two tests commented out:
{noformat}
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running ExampleIT
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.02 s - in ExampleIT
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0{noformat}
> Exceptions in parameterized test sources are ignored
> ----------------------------------------------------
>
> Key: SUREFIRE-1741
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1741
> Project: Maven Surefire
> Issue Type: Bug
> Components: JUnit 5.x support, Maven Failsafe Plugin, Maven Surefire Plugin
> Affects Versions: 3.0.0-M4
> Reporter: Albert Johnston
> Priority: Major
>
> Associated versions on my end at time of testing:
> * JUnit Jupiter 5.5.1
> * JDK 11.0.5
> * Maven 3.6.1
>
> If the following code is run, either for Surefire or Failsafe, only two of the test methods are logged as having been detected at all. The third one is completely ignored as if it didn't exist.
> {code:java}
> public class ExampleTest {
> @Test
> public void shouldRunAndPassAsExpected() {
> }
> public static Stream<Arguments> nothing() {
> return Stream.of(Arguments.arguments());
> }
> @ParameterizedTest
> @MethodSource("nothing")
> public void shouldRunAndFailAsExpected() {
> Assertions.fail();
> }
> static Stream<Arguments> throwException() {
> if (true) throw new RuntimeException("Stop");
> return Stream.of(Arguments.arguments());
> }
> @ParameterizedTest
> @MethodSource("throwException")
> void willNotBeLoggedAsFailingDespiteNotRunning() {
> Assertions.fail();
> }
> }
> {code}
> This is particularly obvious if the first two test methods are commented out, as running mvn test will log that 0 tests were run.
>
> In case it helps, here's the output I'm seeing when running it with the first two tests commented out:
> {noformat}
> [INFO] -------------------------------------------------------
> [INFO] T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running ExampleIT
> [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.02 s - in ExampleIT
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0{noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)