You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Gerold Broser (Jira)" <ji...@apache.org> on 2023/06/16 14:44:00 UTC
[jira] [Comment Edited] (SUREFIRE-2180) Surefire not invoking tests with TestNG DataProvider
[ https://issues.apache.org/jira/browse/SUREFIRE-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733531#comment-17733531 ]
Gerold Broser edited comment on SUREFIRE-2180 at 6/16/23 2:43 PM:
------------------------------------------------------------------
I recently learned at SO that Surefire seems to be a bit picky regarding the (correct–at least what it "thinks" is the correct) version of the test framework dependencies declared in the POM. See the very end of [my answer to _Java unit test run successfully on Intellij but fail on maven_|https://stackoverflow.com/a/76426292/1744774]:
{quote}So, it seems that incompatibilities between JUnit dependencies (in a certain version) and Maven's Surefire plugin (in a certain version) might have caused the original issue.
{quote}
Perhaps something similar applies to this issue as well.
was (Author: geri broser):
I recently learned at SO that Surefire seems to be a bit picky regarding the (correct–at least what it "thinks" is the correct) version of the test framework dependencies declared in the POM. See the very end of my answer to [_Java unit test run successfully on Intellij but fail on maven_](https://stackoverflow.com/a/76426292/1744774):
{quote}So, it seems that incompatibilities between JUnit dependencies (in a certain version) and Maven's Surefire plugin (in a certain version) might have caused the original issue.{quote}
Perhaps something similar applies to this issue as well.
> Surefire not invoking tests with TestNG DataProvider
> ----------------------------------------------------
>
> Key: SUREFIRE-2180
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2180
> Project: Maven Surefire
> Issue Type: Bug
> Components: TestNG support
> Affects Versions: 3.1.2
> Reporter: Javier A. Ortiz
> Priority: Major
>
> I'm having a weird issue I can't figure out. It's driving me insane.
> I have a Maven project for which I'm running the same command both locally and on Jenkins. The environment is the same as far as I can tell:
> - Maven 3.9.2 (On Jenkins: Docker image maven:3.9.2-eclipse-temurin-11)
> - JDK 11 (On Jenkins: Docker image maven:3.9.2-eclipse-temurin-11)
> - TestNG 7.8.0
> Command:
> {code:java}
> mvn -Ddependency.surefire.verbose=10 -Pcoverage verify -B -U -Dgroup=PLATFORM
> {code}
> Relevant POM section:
> {code:java}
> <build>
> <pluginManagement>
> <plugins>
> <plugin>
> <groupId>org.jacoco</groupId>
> <artifactId>jacoco-maven-plugin</artifactId>
> <version>0.8.10</version>
> </plugin>
> ...
> </plugins>
> </pluginManagement>
> ...
> </build>
> ...
> <profiles>
> <profile>
> <id>coverage</id>
> <build>
> <plugins>
> <plugin>
> <groupId>org.jacoco</groupId>
> <artifactId>jacoco-maven-plugin</artifactId>
> <configuration>
> <excludes>
> <exclude>com/XXX/framework/dataproviders/**/*</exclude>
> <exclude>com/XXX/v2/basetest/**/*</exclude>
> <exclude>**/*Exception*</exclude>
> </excludes>
> </configuration>
> <executions>
> <execution>
> <goals>
> <goal>prepare-agent</goal>
> </goals>
> </execution>
> <!-- attached to Maven test phase -->
> <execution>
> <id>report</id>
> <phase>test</phase>
> <goals>
> <goal>report</goal>
> </goals>
> </execution>
> <!-- Add this checking -->
> <execution>
> <id>jacoco-check</id>
> <goals>
> <goal>check</goal>
> </goals>
> <configuration>
> <rules>
> <rule>
> <element>BUNDLE</element>
> <limits>
> <limit>
> <counter>INSTRUCTION</counter>
> <value>COVEREDRATIO</value>
> <minimum>60%</minimum>
> </limit>
> <limit>
> <counter>CLASS</counter>
> <value>MISSEDCOUNT</value>
> <maximum>0</maximum>
> </limit>
> </limits>
> </rule>
> <rule>
> <element>PACKAGE</element>
> <limits>
> <limit>
> <counter>LINE</counter>
> <value>COVEREDRATIO</value>
> <minimum>60%</minimum>
> </limit>
> </limits>
> </rule>
> </rules>
> <skip>${skipTests}</skip>
> </configuration>
> </execution>
> </executions>
> </plugin>
> </plugins>
> </build>
> </profile>
> {code}
> One of the tests that have the issue (only tests with DataProviders) show this pattern in the logs. Enabled verbose to level 10 in hopes that something would show up.
> Locally:
> {code:java}
> [2023-06-06 16:55:46.795] [INFO] [TestClass] Creating TestClass for [ClassImpl class=com.XXX.framework.utils.TimeUtilsTest]
> [2023-06-06 16:55:46.796] [INFO] Method public java.lang.Object[][] com.XXX.framework.utils.TimeUtilsTest.getTimeData() has a @Test annotation but also a return value: ignoring it. Use <suite allow-return-values="true"> to fix this
> [2023-06-06 16:55:46.797] [INFO] [TestClass] Adding method TimeUtilsTest.testToFormattedString(java.util.concurrent.TimeUnit,int,java.lang.String,com.XXX.framework.annotations.TestNameParameter)[pri:0, instance:null] on TestClass class com.XXX.framework.utils.TimeUtilsTest
> {code}
> ...
> {code:java}
> [2023-06-06 16:55:47.316] [INFO] ===== Test class
> com.XXX.framework.utils.TimeUtilsTest
> [2023-06-06 16:55:47.316] [INFO] @Test TimeUtilsTest.testToFormattedString(java.util.concurrent.TimeUnit,int,java.lang.String,com.XXX.framework.annotations.TestNameParameter)[pri:0, instance:com.XXX.framework.utils.TimeUtilsTest@283eb984]
> [2023-06-06 16:55:47.316] [INFO] ======
> {code}
> ...
> {code:java}
> [TestNG] INVOKING: "Surefire test" - com.XXX.framework.utils.TimeUtilsTest.testToFormattedString(java.util.concurrent.TimeUnit,int,java.lang.String,com.XXX.framework.annotations.TestNameParameter)(value(s): NANOSECONDS, 60, "60ns", TestNameParameter(customName=60 NANOSECONDS))
> {code}
> ...
> Test executes and results, etc.
> On Jenkins:
> {code:java}
> [2023-06-06 21:02:29.270] [INFO] [TestClass] Creating TestClass for [ClassImpl class=com.XXX.framework.utils.TimeUtilsTest]
> [2023-06-06 21:02:29.271] [INFO] Method public java.lang.Object[][] com.XXX.framework.utils.TimeUtilsTest.getTimeData() has a @Test annotation but also a return value: ignoring it. Use <suite allow-return-values="true"> to fix this
> [2023-06-06 21:02:29.271] [INFO] [TestClass] Adding method TimeUtilsTest.testToFormattedString(java.util.concurrent.TimeUnit,int,java.lang.String,com.XXX.framework.annotations.TestNameParameter)[pri:0, instance:null] on TestClass class com.XXX.framework.utils.TimeUtilsTest
> {code}
> ...
> {code:java}
> [2023-06-06 21:02:29.529] [INFO] ===== Test class
> com.XXX.framework.utils.TimeUtilsTest
> [2023-06-06 21:02:29.529] [INFO] @Test TimeUtilsTest.testToFormattedString(java.util.concurrent.TimeUnit,int,java.lang.String,com.XXX.framework.annotations.TestNameParameter)[pri:0, instance:com.XXX.framework.utils.TimeUtilsTest@52f9a620]
> [2023-06-06 21:02:29.529] [INFO] ======
> {code}
> I see no other references about the test class in the rest of the trace. Basically, all tests with a DataProvider don't run.
> Maybe someone has a suggestion on what else to look for.
> I've tried the following:
> - Making sure the proper Test annotation is used (TestNG vs. JUnit
> - Remove test groups or any other listener that could be interfering.
> - Downgrade the TestNG version.
> - Pray.
> Here's the investigation on the TestNG side: [https://github.com/testng-team/testng/issues/2924]
> Question in StackOverflow: https://stackoverflow.com/questions/76418882/why-is-testng-not-running-tests-with-dataprovider
--
This message was sent by Atlassian Jira
(v8.20.10#820010)