You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Thomas Diesler (JIRA)" <ji...@codehaus.org> on 2008/04/25 14:26:48 UTC
[jira] Created: (SUREFIRE-486) Exlude tests via an external exlude
file
Exlude tests via an external exlude file
----------------------------------------
Key: SUREFIRE-486
URL: http://jira.codehaus.org/browse/SUREFIRE-486
Project: Maven Surefire
Issue Type: New Feature
Reporter: Thomas Diesler
Similar to Ant, Surefire should support excludes defined in an external file like this
<testExcludes>
<excludeFile>test-excludes-jboss501.txt</excludeFile>
</testExcludes>
The exclude file would look like this
# [JBWS-2135] WebServiceException: Undefined port type
org/jboss/test/ws/jaxws/jbws1822/**
# [JBWS-2136] ClassCastException: com.sun.xml.ws.server.EndpointMessageContextImpl
org/jboss/test/ws/jaxws/namespace/**
# [JBAS-5359] mapped-name is required for context of deployment
org/jboss/test/ws/jaxws/jbws1611/**
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (SUREFIRE-486) Exlude tests via an external
exlude file
Posted by "Thomas Diesler (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/SUREFIRE-486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=133197#action_133197 ]
Thomas Diesler commented on SUREFIRE-486:
-----------------------------------------
Well, we need this feature nevertheless. Folks that also do, please have a look at MCOMPILER-72
> Exlude tests via an external exlude file
> ----------------------------------------
>
> Key: SUREFIRE-486
> URL: http://jira.codehaus.org/browse/SUREFIRE-486
> Project: Maven Surefire
> Issue Type: New Feature
> Reporter: Thomas Diesler
>
> Similar to Ant, Surefire should support excludes defined in an external file like this
> <testExcludes>
> <excludeFile>test-excludes-jboss501.txt</excludeFile>
> </testExcludes>
> The exclude file would look like this
> # [JBWS-2135] WebServiceException: Undefined port type
> org/jboss/test/ws/jaxws/jbws1822/**
> # [JBWS-2136] ClassCastException: com.sun.xml.ws.server.EndpointMessageContextImpl
> org/jboss/test/ws/jaxws/namespace/**
> # [JBAS-5359] mapped-name is required for context of deployment
> org/jboss/test/ws/jaxws/jbws1611/**
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Closed: (SUREFIRE-486) Exlude tests via an external exlude
file
Posted by "Dan Fabulich (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/SUREFIRE-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dan Fabulich closed SUREFIRE-486.
---------------------------------
Resolution: Won't Fix
I don't think that's the feature you want; there's a better way to do it. JUnit 4 includes a @Ignore annotiation; TestNG allows you to have your tests throw a SkipException which does something similar. That will keep the tests in the report, but mark them as "yellow" or "skipped".
If we excluded known bugs entirely, they would simply disappear from the report.
> Exlude tests via an external exlude file
> ----------------------------------------
>
> Key: SUREFIRE-486
> URL: http://jira.codehaus.org/browse/SUREFIRE-486
> Project: Maven Surefire
> Issue Type: New Feature
> Reporter: Thomas Diesler
>
> Similar to Ant, Surefire should support excludes defined in an external file like this
> <testExcludes>
> <excludeFile>test-excludes-jboss501.txt</excludeFile>
> </testExcludes>
> The exclude file would look like this
> # [JBWS-2135] WebServiceException: Undefined port type
> org/jboss/test/ws/jaxws/jbws1822/**
> # [JBWS-2136] ClassCastException: com.sun.xml.ws.server.EndpointMessageContextImpl
> org/jboss/test/ws/jaxws/namespace/**
> # [JBAS-5359] mapped-name is required for context of deployment
> org/jboss/test/ws/jaxws/jbws1611/**
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Reopened: (SUREFIRE-486) Exlude tests via an external exlude
file
Posted by "Thomas Diesler (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/SUREFIRE-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Diesler reopened SUREFIRE-486:
-------------------------------------
Binding the execution of a test to the test source code is not a good idea. Also your suggested solution would introduce a dependency to a particular testing framework/version
Suppose you have a test suite that runs against various environments. In environment A the test is expected to pass in environment B the test should be excluded. Switching the exclude file depending on environment is common practice in various TCKs (Test Compatibility Kit)
Please have a look at http://jbws.dyndns.org/mediawiki/index.php?title=WSF for more background on this.
> Exlude tests via an external exlude file
> ----------------------------------------
>
> Key: SUREFIRE-486
> URL: http://jira.codehaus.org/browse/SUREFIRE-486
> Project: Maven Surefire
> Issue Type: New Feature
> Reporter: Thomas Diesler
>
> Similar to Ant, Surefire should support excludes defined in an external file like this
> <testExcludes>
> <excludeFile>test-excludes-jboss501.txt</excludeFile>
> </testExcludes>
> The exclude file would look like this
> # [JBWS-2135] WebServiceException: Undefined port type
> org/jboss/test/ws/jaxws/jbws1822/**
> # [JBWS-2136] ClassCastException: com.sun.xml.ws.server.EndpointMessageContextImpl
> org/jboss/test/ws/jaxws/namespace/**
> # [JBAS-5359] mapped-name is required for context of deployment
> org/jboss/test/ws/jaxws/jbws1611/**
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Closed: (SUREFIRE-486) Exlude tests via an external exlude
file
Posted by "Dan Fabulich (JIRA)" <ji...@codehaus.org>.
[ http://jira.codehaus.org/browse/SUREFIRE-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dan Fabulich closed SUREFIRE-486.
---------------------------------
Resolution: Won't Fix
That page doesn't have anything to do with testing, though I'm quite familiar with the idea that some tests may be expected to fail on some platforms but pass on other platforms.
TestNG allows you to specify an external suite file which configures which tests will be skipped. You can also use code to decide when to throw a skip exception. You can even dynamically query your bug DB to find out whether a bug is still unresolved, and skip only if the bug is open. [Of course, it may not be a good idea to make your tests that complex. ;-)]
Your remark about being "tied to a testing framework" is strange, because Surefire only runs tests that are tied to a testing framework: it will run JUnit tests (which are "tied" to JUnit) or it will run TestNG tests (which are "tied" to TestNG). The @Ignore/Skip feature is not specific to either framework; you can use it in either, but if you want to switch frameworks, you have to re-code all of your tests.
More generally, sometimes people file feature requests to Surefire that are appropriately targeted to the underlying test framework. ["I wish JUnit had feature X. But the JUnit guys do releases less than once a year! Maybe I could ask the Surefire guys to do it for me instead."]
This is one of those feature requests that not only *can* be solved by the underlying test framework, it already *has* been solved in both of the test frameworks we support.
> Exlude tests via an external exlude file
> ----------------------------------------
>
> Key: SUREFIRE-486
> URL: http://jira.codehaus.org/browse/SUREFIRE-486
> Project: Maven Surefire
> Issue Type: New Feature
> Reporter: Thomas Diesler
>
> Similar to Ant, Surefire should support excludes defined in an external file like this
> <testExcludes>
> <excludeFile>test-excludes-jboss501.txt</excludeFile>
> </testExcludes>
> The exclude file would look like this
> # [JBWS-2135] WebServiceException: Undefined port type
> org/jboss/test/ws/jaxws/jbws1822/**
> # [JBWS-2136] ClassCastException: com.sun.xml.ws.server.EndpointMessageContextImpl
> org/jboss/test/ws/jaxws/namespace/**
> # [JBAS-5359] mapped-name is required for context of deployment
> org/jboss/test/ws/jaxws/jbws1611/**
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira