You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Peter Prochazka (JIRA)" <ji...@codehaus.org> on 2011/08/26 12:33:32 UTC
[jira] Created: (SUREFIRE-765) Testng xml bug no cause build
failure
Testng xml bug no cause build failure
-------------------------------------
Key: SUREFIRE-765
URL: https://jira.codehaus.org/browse/SUREFIRE-765
Project: Maven Surefire
Issue Type: Bug
Components: Maven Surefire Plugin
Affects Versions: 2.9
Environment: tested with same result on
windows 7
ubuntu linux
Both use Oracle JDK 6
Reporter: Peter Prochazka
Attachments: sample.zip
During usage of surefire report we see following behavior:
# We have maven project with TestNG framework
# I create wrong testng.xml file (invalid listed test)
# we run maven test
# and we see
## Exception from testNG - this is expected
## but result is everything *OK*
but expected behaviour is *FAILURE*
pom.xml surefire settings
{code:xml}
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.9</version>
<configuration>
<forkMode>once</forkMode>
<redirectTestOutputToFile>true</redirectTestOutputToFile>
<trimStackTrace>false</trimStackTrace>
<suiteXmlFiles>
<suiteXmlFile>testng.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-testng</artifactId>
<version>2.9</version>
</dependency>
</dependencies>
</plugin>
{code}
*mvn clean test*
Result is exception from testng which is OK
but surefire reports SUCCESS
{code}
[WARNING] Using platform encoding (Cp1250 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] Copying 0 resource
[INFO] [compiler:testCompile {execution: default-testCompile}]
[WARNING] File encoding has not been set, using platform encoding Cp1250, i.e. build is platform dependent!
[INFO] Compiling 1 source file to C:\workspace\sample\target\test-classes
[INFO] [surefire:test {execution: default-test}]
[INFO] Surefire report directory: C:\workspace\sample\target\surefire-reports
[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running TestSuite
org.apache.maven.surefire.util.SurefireReflectionException: java.lang.reflect.InvocationTargetException; nested exception is java.lang.reflect.Invocat
ionTargetException: null
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:172)
at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:104)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:70)
Caused by: org.testng.TestNGException:
Cannot find class in classpath: sample.NotImplmentedTest
at org.testng.xml.XmlClass.loadClass(XmlClass.java:76)
at org.testng.xml.XmlClass.init(XmlClass.java:68)
at org.testng.xml.XmlClass.<init>(XmlClass.java:54)
at org.testng.xml.TestNGContentHandler.startElement(TestNGContentHandler.java:516)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanStartElement(XMLDocumentFragmentScannerImpl.java:1343)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2755)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:511)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)
at javax.xml.parsers.SAXParser.parse(SAXParser.java:395)
at javax.xml.parsers.SAXParser.parse(SAXParser.java:198)
at org.testng.xml.SuiteXmlParser.parse(SuiteXmlParser.java:17)
at org.testng.xml.SuiteXmlParser.parse(SuiteXmlParser.java:10)
at org.testng.xml.Parser.parse(Parser.java:170)
at org.testng.TestNG.initializeSuitesAndJarFile(TestNG.java:299)
at org.testng.TestNG.run(TestNG.java:972)
at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:122)
at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:92)
at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:101)
... 9 more
Results :
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO] [jar:jar {execution: default-jar}]
[INFO] Building jar: C:\workspace\sample\target\sample-0.0.1-SNAPSHOT.jar
[INFO] [install:install {execution: default-install}]
[INFO] Installing C:\workspace\sample\target\sample-0.0.1-SNAPSHOT.jar to C:\Users\mamut\.m2\repository\sample\sample\0.0.1-SNAPSHOT\sample-0.0.1-SNAP
SHOT.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3 seconds
[INFO] Finished at: Fri Aug 26 12:25:30 CEST 2011
[INFO] Final Memory: 14M/35M
[INFO] ------------------------------------------------------------------------
PS C:\workspace\sample>
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-765) Testng xml bug no cause build failure
Posted by "Holger Bartnick (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=295685#comment-295685 ]
Holger Bartnick commented on SUREFIRE-765:
------------------------------------------
we´ve got a problem with that:
we defined in our basic-pom multiple profiles for "nightly build" and "developer"-builds etc
having different specifications for suiteXmlFiles/suitexmlFile.
in the underlying poms with no test file now an error raises when no suite-file is available. :(
> Testng xml bug no cause build failure
> -------------------------------------
>
> Key: SUREFIRE-765
> URL: https://jira.codehaus.org/browse/SUREFIRE-765
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Plugin
> Affects Versions: 2.9
> Environment: tested with same result on
> windows 7
> ubuntu linux
> Both use Oracle JDK 6
> Reporter: Peter Prochazka
> Assignee: Kristian Rosenvold
> Fix For: 2.11
>
> Attachments: sample.zip
>
>
> During usage of surefire report we see following behavior:
> # We have maven project with TestNG framework
> # I create wrong testng.xml file (invalid listed test)
> # we run maven test
> # and we see
> ## Exception from testNG - this is expected
> ## but result is everything *OK*
> but expected behaviour is *FAILURE*
> pom.xml surefire settings
> {code:xml}
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-surefire-plugin</artifactId>
> <version>2.9</version>
> <configuration>
> <forkMode>once</forkMode>
> <redirectTestOutputToFile>true</redirectTestOutputToFile>
> <trimStackTrace>false</trimStackTrace>
> <suiteXmlFiles>
> <suiteXmlFile>testng.xml</suiteXmlFile>
> </suiteXmlFiles>
> </configuration>
> <dependencies>
> <dependency>
> <groupId>org.apache.maven.surefire</groupId>
> <artifactId>surefire-testng</artifactId>
> <version>2.9</version>
> </dependency>
> </dependencies>
> </plugin>
> {code}
> *mvn clean test*
> Result is exception from testng which is OK
> but surefire reports SUCCESS
> {code}
> [WARNING] Using platform encoding (Cp1250 actually) to copy filtered resources, i.e. build is platform dependent!
> [INFO] Copying 0 resource
> [INFO] [compiler:testCompile {execution: default-testCompile}]
> [WARNING] File encoding has not been set, using platform encoding Cp1250, i.e. build is platform dependent!
> [INFO] Compiling 1 source file to C:\workspace\sample\target\test-classes
> [INFO] [surefire:test {execution: default-test}]
> [INFO] Surefire report directory: C:\workspace\sample\target\surefire-reports
> [INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Running TestSuite
> org.apache.maven.surefire.util.SurefireReflectionException: java.lang.reflect.InvocationTargetException; nested exception is java.lang.reflect.Invocat
> ionTargetException: null
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
> at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
> at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:172)
> at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:104)
> at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:70)
> Caused by: org.testng.TestNGException:
> Cannot find class in classpath: sample.NotImplmentedTest
> at org.testng.xml.XmlClass.loadClass(XmlClass.java:76)
> at org.testng.xml.XmlClass.init(XmlClass.java:68)
> at org.testng.xml.XmlClass.<init>(XmlClass.java:54)
> at org.testng.xml.TestNGContentHandler.startElement(TestNGContentHandler.java:516)
> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
> at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanStartElement(XMLDocumentFragmentScannerImpl.java:1343)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2755)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:511)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
> at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)
> at javax.xml.parsers.SAXParser.parse(SAXParser.java:395)
> at javax.xml.parsers.SAXParser.parse(SAXParser.java:198)
> at org.testng.xml.SuiteXmlParser.parse(SuiteXmlParser.java:17)
> at org.testng.xml.SuiteXmlParser.parse(SuiteXmlParser.java:10)
> at org.testng.xml.Parser.parse(Parser.java:170)
> at org.testng.TestNG.initializeSuitesAndJarFile(TestNG.java:299)
> at org.testng.TestNG.run(TestNG.java:972)
> at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:122)
> at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:92)
> at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:101)
> ... 9 more
> Results :
> Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
> [INFO] [jar:jar {execution: default-jar}]
> [INFO] Building jar: C:\workspace\sample\target\sample-0.0.1-SNAPSHOT.jar
> [INFO] [install:install {execution: default-install}]
> [INFO] Installing C:\workspace\sample\target\sample-0.0.1-SNAPSHOT.jar to C:\Users\mamut\.m2\repository\sample\sample\0.0.1-SNAPSHOT\sample-0.0.1-SNAP
> SHOT.jar
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESSFUL
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 3 seconds
> [INFO] Finished at: Fri Aug 26 12:25:30 CEST 2011
> [INFO] Final Memory: 14M/35M
> [INFO] ------------------------------------------------------------------------
> PS C:\workspace\sample>
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] (SUREFIRE-765) Testng xml bug no cause build failure
Posted by "Kristian Rosenvold (JIRA)" <ji...@codehaus.org>.
[ https://jira.codehaus.org/browse/SUREFIRE-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Rosenvold closed SUREFIRE-765.
---------------------------------------
Resolution: Fixed
Fix Version/s: 2.11
Assignee: Kristian Rosenvold
This issue was fixed for 2.11
> Testng xml bug no cause build failure
> -------------------------------------
>
> Key: SUREFIRE-765
> URL: https://jira.codehaus.org/browse/SUREFIRE-765
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Plugin
> Affects Versions: 2.9
> Environment: tested with same result on
> windows 7
> ubuntu linux
> Both use Oracle JDK 6
> Reporter: Peter Prochazka
> Assignee: Kristian Rosenvold
> Fix For: 2.11
>
> Attachments: sample.zip
>
>
> During usage of surefire report we see following behavior:
> # We have maven project with TestNG framework
> # I create wrong testng.xml file (invalid listed test)
> # we run maven test
> # and we see
> ## Exception from testNG - this is expected
> ## but result is everything *OK*
> but expected behaviour is *FAILURE*
> pom.xml surefire settings
> {code:xml}
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-surefire-plugin</artifactId>
> <version>2.9</version>
> <configuration>
> <forkMode>once</forkMode>
> <redirectTestOutputToFile>true</redirectTestOutputToFile>
> <trimStackTrace>false</trimStackTrace>
> <suiteXmlFiles>
> <suiteXmlFile>testng.xml</suiteXmlFile>
> </suiteXmlFiles>
> </configuration>
> <dependencies>
> <dependency>
> <groupId>org.apache.maven.surefire</groupId>
> <artifactId>surefire-testng</artifactId>
> <version>2.9</version>
> </dependency>
> </dependencies>
> </plugin>
> {code}
> *mvn clean test*
> Result is exception from testng which is OK
> but surefire reports SUCCESS
> {code}
> [WARNING] Using platform encoding (Cp1250 actually) to copy filtered resources, i.e. build is platform dependent!
> [INFO] Copying 0 resource
> [INFO] [compiler:testCompile {execution: default-testCompile}]
> [WARNING] File encoding has not been set, using platform encoding Cp1250, i.e. build is platform dependent!
> [INFO] Compiling 1 source file to C:\workspace\sample\target\test-classes
> [INFO] [surefire:test {execution: default-test}]
> [INFO] Surefire report directory: C:\workspace\sample\target\surefire-reports
> [INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Running TestSuite
> org.apache.maven.surefire.util.SurefireReflectionException: java.lang.reflect.InvocationTargetException; nested exception is java.lang.reflect.Invocat
> ionTargetException: null
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
> at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
> at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:172)
> at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:104)
> at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:70)
> Caused by: org.testng.TestNGException:
> Cannot find class in classpath: sample.NotImplmentedTest
> at org.testng.xml.XmlClass.loadClass(XmlClass.java:76)
> at org.testng.xml.XmlClass.init(XmlClass.java:68)
> at org.testng.xml.XmlClass.<init>(XmlClass.java:54)
> at org.testng.xml.TestNGContentHandler.startElement(TestNGContentHandler.java:516)
> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
> at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanStartElement(XMLDocumentFragmentScannerImpl.java:1343)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2755)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648)
> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:511)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
> at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)
> at javax.xml.parsers.SAXParser.parse(SAXParser.java:395)
> at javax.xml.parsers.SAXParser.parse(SAXParser.java:198)
> at org.testng.xml.SuiteXmlParser.parse(SuiteXmlParser.java:17)
> at org.testng.xml.SuiteXmlParser.parse(SuiteXmlParser.java:10)
> at org.testng.xml.Parser.parse(Parser.java:170)
> at org.testng.TestNG.initializeSuitesAndJarFile(TestNG.java:299)
> at org.testng.TestNG.run(TestNG.java:972)
> at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:122)
> at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:92)
> at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:101)
> ... 9 more
> Results :
> Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
> [INFO] [jar:jar {execution: default-jar}]
> [INFO] Building jar: C:\workspace\sample\target\sample-0.0.1-SNAPSHOT.jar
> [INFO] [install:install {execution: default-install}]
> [INFO] Installing C:\workspace\sample\target\sample-0.0.1-SNAPSHOT.jar to C:\Users\mamut\.m2\repository\sample\sample\0.0.1-SNAPSHOT\sample-0.0.1-SNAP
> SHOT.jar
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESSFUL
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 3 seconds
> [INFO] Finished at: Fri Aug 26 12:25:30 CEST 2011
> [INFO] Final Memory: 14M/35M
> [INFO] ------------------------------------------------------------------------
> PS C:\workspace\sample>
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira